Hypothesis Testing: The p-value

What is hypothesis testing?

Hypothesis testing asks: "Is the observed result significantly different from what we'd expect by chance?" We set up a null hypothesis (the default assumption), compute how unlikely our data would be under that assumption, and decide whether to reject it.

Step-by-step: Is this coin fair?

Step 1 — State the hypotheses.

$H_0$: $p = 0.5$ (the coin is fair — null hypothesis).

$H_a$: $p \neq 0.5$ (the coin is biased — alternative hypothesis).

Step 2 — Collect data. 100 flips, 60 heads. $\hat{p} = 60/100 = 0.60$.

Step 3 — Compute the test statistic. Under $H_0$, $\hat{p}$ has mean $p_0 = 0.5$ and standard error $\sqrt{p_0(1-p_0)/n}$:

$z = \frac{\hat{p} - p_0}{\sqrt{p_0(1-p_0)/n}} = \frac{0.60 - 0.50}{\sqrt{0.25/100}} = \frac{0.10}{0.05} = 2.0$

Step 4 — Find the p-value. This is the probability of observing a test statistic as extreme as $|z| = 2.0$ under $H_0$ (two-tailed):

$p\text{-value} = P(|Z| > 2.0) = 2 \times P(Z > 2.0) = 2 \times 0.0228 = 0.0456$

Step 5 — Compare to significance level $\alpha = 0.05$.

$p\text{-value} = 0.0456 < 0.05 = \alpha$

Decision: Reject $H_0$. There is statistically significant evidence that the coin is biased.

What the p-value means

The p-value is the probability of seeing results at least as extreme as ours, assuming the null hypothesis is true. A small p-value ($< \alpha$) means the data is unlikely under $H_0$ — evidence against it.

• p-value < 0.05: "statistically significant" (reject $H_0$)
• p-value ≥ 0.05: "not significant" (fail to reject $H_0$ — this does NOT prove $H_0$ is true)

Try it in the visualization

Adjust the number of flips and heads. The normal curve shows the test statistic, rejection region (red shading), and p-value. When p < α, the decision flips to "reject."