Linear Regression: Line of Best Fit

What is linear regression?

Linear regression finds the line $\hat{y} = mx + b$ that minimizes the sum of squared residuals — the squared vertical distances from each point to the line.

Step-by-step: {(1,2), (2,4), (3,5), (4,4), (5,5)}

Step 1 — Compute sums. $n = 5$, $\sum x = 15$, $\sum y = 20$, $\sum xy = 67$, $\sum x^2 = 55$.

Step 2 — Slope:

$m = \frac{n\sum xy - \sum x \sum y}{n\sum x^2 - (\sum x)^2} = \frac{5(67) - 15(20)}{5(55) - 225} = \frac{335 - 300}{275 - 225} = \frac{35}{50} = 0.7$

Step 3 — Intercept: $b = \bar{y} - m\bar{x} = 4 - 0.7(3) = 1.9$

Result: $\hat{y} = 0.7x + 1.9$

Step 4 — $R^2$: Measures how well the line fits. $R^2 = 1$ is perfect; $R^2 = 0$ means the line explains nothing.

Residuals

A residual is $e_i = y_i - \hat{y}_i$ (actual minus predicted). The regression line minimizes $\sum e_i^2$.

Try it in the visualization

Scatter plot with the regression line. Residual lines connect each point to the line. Toggle "squared residuals" to see them as literal squares — the line minimizes total square area.