The cheat sheet

Read a symbol as an instruction about an experiment. Worked notation problems show how several instructions combine in a full question.

Events and conditioning

notation	read it as
$\Omega$, $\omega$	sample space; one outcome in it
$A\subseteq\Omega$	event A is a set of possible outcomes
$A\cap B$, $A\cup B$	both events; at least one event
$\lvert A\rvert$	number of outcomes in a finite event
$\Pr(A)$	probability of A
$\Pr(A\mid B)$	probability of A after conditioning on B

For finite equally likely outcomes, $\Pr(A)=|A|/|\Omega|$. More generally, add the probability mass assigned to the outcomes in the event.

Probabilities are nonnegative, the whole space has probability 1, and probabilities add for a countable collection of pairwise disjoint events. In particular:

$\Pr(A^c)=1-\Pr(A),\qquad\Pr(A\cup B)\le\Pr(A)+\Pr(B).$

$\Pr(A\mid B)=\frac{\Pr(A\cap B)}{\Pr(B)},\qquad\Pr(B)>0.$

Events A and B are independent when $\Pr(A\cap B)=\Pr(A)\Pr(B)$.

One variable, three representations

name	definition	how to get a probability
PMF	$p_X(x)=\Pr(X=x)$	sum masses at relevant values
PDF	$f_X(x)$: probability per unit of x	integrate over a region
CDF	$F_X(x)=\Pr(X\le x)$	subtract accumulated probabilities

A density can exceed 1; its total area must be 1. A CDF is nondecreasing, with limits 0 and 1 at the ends of the real line. For any distribution:

$\Pr(a<X\le b)=F_X(b)-F_X(a).$

The lower endpoint is excluded. For a distribution with a density, endpoints carry no probability and the distinction disappears. In our smooth examples, $f_X=F_X'$ and $F_X(x)=\int_{-\infty}^{x}f_X(t)\,dt$.

Expectations and variance

$\mathbb E[g(X)]=\sum_x g(x)p_X(x).$

Evaluate g at each possible x first, then weight and add. For a density, use $\int g(x)f_X(x)\,dx$ instead.

With finite moments:

$\mathbb E[aX+bY+c]=a\mathbb E[X]+b\mathbb E[Y]+c.$

This does not require independence. Nonlinear operations generally cannot move outside the expectation: $\mathbb E[X^2]$ is usually different from $(\mathbb E[X])^2$.

$\operatorname{Var}(X)=\mathbb E[(X-\mathbb E[X])^2]=\mathbb E[X^2]-(\mathbb E[X])^2.$

$\operatorname{Var}(aX+b)=a^2\operatorname{Var}(X).$

$\operatorname{Var}(X+Y)=\operatorname{Var}(X)+\operatorname{Var}(Y)+2\operatorname{Cov}(X,Y).$

Independent variables have zero covariance. Zero covariance alone is sufficient for adding these two variances; it does not imply independence. Standard deviation is $\sqrt{\operatorname{Var}(X)}$, in the original units.

Two variables

operation	discrete formula	physical operation
joint	$p_{X,Y}(x,y)$	probability of a pair
marginal	$p_X(x)=\sum_y p_{X,Y}(x,y)$	add a row
conditional	$p_{Y\mid X}(y\mid x)=p_{X,Y}(x,y)/p_X(x)$	keep a row, divide by its total

The conditional PMF formula requires $p_X(x)>0$. For joint densities, $f_{Y\mid X}(y\mid x)=f_{X,Y}(x,y)/f_X(x)$ when $f_X(x)>0$. This is a ratio of densities, not probabilities of exact continuous values.

Bayes’ rule reverses which quantity is conditioned on:

$p_{X\mid Y}(x\mid y)=\frac{p_{Y\mid X}(y\mid x)p_X(x)}{p_Y(y)}.$

$\operatorname{Cov}(X,Y)=\mathbb E[XY]-\mathbb E[X]\mathbb E[Y].$

Condition first, then average back over the groups:

$\mathbb E[Y]=\sum_x\mathbb E[Y\mid X=x]p_X(x).$

For continuous X, replace the sum with an integral using its density. The analogous averaging of conditional probabilities is the law of total probability.

Common distributions

notation	experiment	mean	variance
$\operatorname{Ber}(p)$	1 with probability p; otherwise 0	p	p(1−p)
$\operatorname{Bin}(n,p)$	successes in n independent identical Bernoulli trials	np	np(1−p)
$\operatorname{Unif}[a,b]$	uniform point in an interval, a<b	(a+b)/2	(b−a)²/12
$\mathcal N(\mu,\sigma^2)$	normal with mean μ, standard deviation σ	μ	σ²

The standard normal CDF is $\Phi$. It has no elementary closed form; numerical software computes it accurately. For $X\sim\mathcal N(\mu,\sigma^2)$, standardize with $Z=(X-\mu)/\sigma$ before using Φ.

Notation you will see in ML

• $X_1,\ldots,X_n\overset{\mathrm{iid}}{\sim}p$: independent draws from the same distribution p.
• $\mathbf 1\{A\}$: 1 when A occurs, otherwise 0. Its expectation is $\Pr(A)$.
• $\mathbb E_{x\sim p}[\log q_\theta(x)]$: draw x from p, compute its log score under $q_\theta$, average.
• $\operatorname{argmax}_\theta$: return the parameter choice with the largest score. $\operatorname{max}_\theta$ returns the score itself.

Try mixed practice →