Least-Squares Regression Line
Derive and apply the LSRL ŷ = a + bx where b = r(sy/sx) and a = ȳ − bx̄. Interpret the slope and y-intercept in context. Understand that the LSRL minimizes the sum of squared residuals.
A lesson of AP Statistics, in Exploring Two-Variable Data, about 10 minutes.
Least-Squares Regression
By the end of this lesson, you will be able to find and interpret the LSRL equation, use it to make predictions, and understand what makes the LSRL the "best" line.
The Least-Squares Regression Line (LSRL)
When there is a linear relationship between two quantitative variables, we can fit a line to predict y from x:
$$\hat{y} = a + bx$$
where $\hat{y}$ is the predicted value of y, $a$ is the y-intercept, and $b$ is the slope.
Why "least-squares"? The LSRL is the line that minimizes the sum of squared residuals — the sum of $(y_i - \hat{y}_i)^2$. No other line produces a smaller total squared error.
Finding the LSRL
The formulas for the slope and intercept:
$$b = r \cdot \frac{s_y}{s_x} \qquad a = \bar{y} - b\bar{x}$$
where r is the correlation coefficient, $s_x$ and $s_y$ are the standard deviations of x and y, and $\bar{x}$, $\bar{y}$ are the means.
Key fact: The LSRL always passes through the point $(\bar{x}, \bar{y})$.
Interpreting the LSRL
Slope ($b$): "For each 1-unit increase in [x], the predicted [y] changes by $b$ [units of y], on average."
Intercept ($a$): "When [x] = 0, the predicted [y] is $a$ [units]." Often not meaningful in context (extrapolation to x = 0 may be unrealistic).
Example: $\hat{\text{score}} = 40 + 9 \cdot (\text{hours studied})$
- Slope: Each additional hour studied is associated with a predicted 9-point increase in score.
- Intercept: A student who studies 0 hours is predicted to score 40.
Making Predictions
Interpolation: predicting within the observed range of x. Generally reliable.
Extrapolation: predicting outside the observed range of x. Unreliable — the linear relationship may not extend beyond the observed data.
Example: If the above equation was fit using students who studied 1–6 hours, predicting for a student who studies 15 hours (score = 175) would be extrapolation — scores can't exceed 100, so the linear model breaks down outside its range.