Law of Large Numbers: Convergence to the Mean

The Law of Large Numbers (LLN)

As the number of trials increases, the sample mean converges to the expected value. More trials → closer to the theoretical average. This is the mathematical foundation of why statistics works.

Step-by-step example: rolling a fair die

Step 1 — Compute the expected value.

$E(X) = \frac{1 + 2 + 3 + 4 + 5 + 6}{6} = \frac{21}{6} = 3.5$

Step 2 — Roll and compute running averages:

• After 1 roll (got a 4): average $= 4.0$ (far from 3.5)
• After 10 rolls: average $\approx 3.2$ (closer)
• After 100 rolls: average $\approx 3.48$ (quite close)
• After 1,000 rolls: average $\approx 3.502$ (very close)
• After 10,000 rolls: average $\approx 3.4997$ (nearly exact)

What the LLN does NOT say

• ❌ "If you get 5 heads in a row, tails is 'due'." (The gambler's fallacy — each flip is independent!)
• ❌ "The average will be exactly 3.5 after enough rolls." (It converges, but never equals exactly.)
• ✅ "The average will get arbitrarily close to 3.5 as $n$ grows."

Why this matters

The LLN is why casinos are profitable (they play millions of games at a slight edge), why insurance works (averaging over many policyholders), and why polls with larger sample sizes are more accurate.

Try it in the visualization

Watch the running average line stabilize as rolls accumulate. Early on, it swings wildly. After thousands of rolls, it locks onto 3.5. Toggle "5 independent trials" to see multiple lines all converging to the same value.