Chance is a fraction

The bag of everything that can happen

Roll a die. Before it lands, exactly six things can happen. Collect them in a set and call it Ω (“omega” — the bag of everything that can happen):

$$\Omega=\{1,2,3,4,5,6\}$$

A single outcome gets the small letter ω, and ω ∈ Ω just says “this outcome is one of the things in the bag.” An event is a question about the outcome, written as the set of outcomes that answer yes: “the roll is even” is the event A = {2, 4, 6}. We write A ⊆ Ω, “A is a piece of Ω.”

Probability is counting

When every outcome is equally likely, probability is not a new idea:

$$\Pr(A)=\frac{|A|}{|\Omega|}$$

Pr(A) is the chance that A happens, and |A| means “how many outcomes are in A.” Count the outcomes you care about, divide by everything that can happen.

$$\Pr(\text{even})=\frac{|\{2,4,6\}|}{|\{1,2,3,4,5,6\}|}=\frac36=\frac12$$

One picture to carry

Probability behaves like mass. Spread a fixed lump — total 1 — over the outcomes; an event’s probability is how much sits on it. A fair die gets 1/6 per face, and counting is just the even spread. A loaded die — Pr(6) = 1/2, and 1/10 on each other face — is no longer counting, but it is the same picture, and everything below still holds.

The rules probability obeys

• Never negative. You cannot have less than none of something, so Pr(A) ≥ 0.
• The whole bag is certain. Pr(Ω) = 1 — everything is somewhere.
• If no overlap, add. If A ∩ B (“the outcomes in both A and B”) is empty, then Pr(A ∪ B) (“the outcomes in A or B”) is just Pr(A) + Pr(B) — separate pieces, separate counts.

These rules also apply beyond finite examples. Formally, the additivity rule extends to any countable collection of pairwise disjoint events. We only need finite collections for the counting examples here.

What follows from those rules

• The complement. $\Pr(A^c) = 1 - \Pr(A)$ — what isn’t on A is on the rest: Pr(not even) = 1 − 1/2 = 1/2.
• A piece can’t exceed the whole. If A ⊆ B, then Pr(A) ≤ Pr(B): {6} ⊆ {5, 6}, and 1/6 ≤ 2/6.
• The union bound. $\Pr(A \cup B) \le \Pr(A) + \Pr(B)$ — adding the two counts any shared outcomes twice: Pr(even or at least 5) = 4/6, while the sum promises at most 3/6 + 2/6 = 5/6, because 6 was counted twice. (This one is everywhere in ML.)

Go deeper: deriving the free facts

Each is the add rule in disguise. A and “not A” don’t overlap and together fill the bag, so Pr(A) + Pr(not A) = 1. B splits into A and “B without A,” which don’t overlap, so Pr(B) = Pr(A) + Pr(B without A) ≥ Pr(A). And Pr(A ∪ B) = Pr(A) + Pr(B without A) ≤ Pr(A) + Pr(B), since shrinking an event can’t raise its probability.

Observing shrinks the world

Someone rolls the die, peeks, and tells you only: “it’s even.” What did that news do?

It threw away every outcome outside “even.” Now you count again, inside the smaller world. For an event B with positive probability, conditional probability is:

$$\Pr(A\mid B) = \Pr(A \cap B) / \Pr(B)$$

Read Pr(A | B) as “the chance of A, given that B happened”: on top, the still-alive outcomes you want; on the bottom, everything still alive. Dividing by Pr(B) rescales the smaller world so it totals 1 again. (This only makes sense when Pr(B) > 0.)

Fix one question, A = “at least 5” = {5, 6}, and try two different pieces of news side by side. Each row is the same move: throw away the dead outcomes, count again.

the news	still alive	of those, in A	chance of A
(none yet)	1 2 3 4 5 6	5, 6	2/6 = 1/3
”it’s even”	2 4 6	6	1/3 — didn’t move
”5 or less”	1 2 3 4 5	5	1/5 — moved

Row two, in formula form: $\Pr(A\mid \text{even}) = (1/6) / (3/6) = 1/3$ — on top the one survivor in A, on the bottom the three survivors, both measured in the old world. The news changed nothing: $\Pr(A\mid B) = \Pr(A)$. That is independence — B carries no information about A. Multiply both sides of the shrink formula by Pr(B): $\Pr(A \cap B) = \Pr(A\mid B) \times \Pr(B)$ — and since the news changed nothing, $\Pr(A \cap B) = \Pr(A) \times \Pr(B)$.

Row three is the other case: same A, different news, and the fraction moved. Dependence is just this — the shrink moved the fraction.

Pick an event, then observe another one. The world shrinks; the counting stays the same.

Try it

Observe “even”, then “5 or less”, and watch which outcomes get thrown away. The fraction for A sometimes moves (dependence) and sometimes stays put (independence).

Next

An event is a set of outcomes. Its probability is the mass assigned to that set; it equals a fraction of the outcome count when outcomes are equally likely. The next move is to attach a number to each outcome — and suddenly there are tables, curves, and densities.