Linear Regression Models

A scatterplot describes a relationship; a correlation measures how linear it is. The next step is to use the relationship: to predict the response for a given value of the explanatory variable. When the pattern is linear, a straight line does that.

The regression line

A regression line is a line that describes how the response $y$ changes, on average, as the explanatory variable $x$ changes. In statistics it is written

$$\hat{y} = a + bx,$$

where $a$ is the intercept and $b$ is the slope, the same kind of equation as $y = mx + b$ from algebra. Watch the letters: here $b$ is the slope (algebra's $m$) and $a$ is the intercept (algebra's $b$), written first. The hat on $\hat{y}$ ("y-hat") says that the output is a predicted value, not an observed one. Real data points scatter above and below the line; the line gives the typical value for each $x$. (This equation, $\hat{y} = a + bx$, is on the reference sheet.)

Where does the line come from? Usually from technology, which fits the line to the data. Topic 5.5 explains how the best line is chosen. For now, take the equation as given and learn to use it.

Making a prediction

Interpreting the slope

The slope is the change in the predicted value for each one-unit increase in $x$. Its units are units of $y$ per unit of $x$. Use this template, filling in context:

"For each additional [unit of $x$], the predicted [$y$] increases (decreases) by about $|b|$ [units of $y$]."

For the sleep line: for each additional hour of sleep, the predicted reaction time decreases by about 11.4 milliseconds. The word "predicted" matters. The slope does not say that every student who sleeps one more hour reacts exactly 11.4 milliseconds faster; it describes the trend of the predictions.

The slope always has the same sign as the correlation: a negative association gives a falling line.

Interpreting the intercept, when it makes sense

The intercept $a$ is the predicted $y$ when $x = 0$. Interpret it only if $x = 0$ is a sensible value inside, or very near, the data.

Compare a line for children aged 5 to 12, $\hat{y} = 76 + 6.2x$ for height in centimeters. The intercept would describe a newborn, but no newborns are in the data, and children do not grow at a steady rate from birth. So the intercept has no useful meaning here.

Extrapolation

A line is fit to the $x$ values in the data. Using it far outside that range is extrapolation, and it is unreliable: nothing in the data says the pattern continues. The phone line predicts $620 - 95(8) = -140$ dollars for an 8-year-old phone, an impossible price. The sleep line predicts about 243 milliseconds for 13 hours of sleep, far faster than any student in the data, who all slept 4.5 to 9.5 hours. Both numbers come out of the equation; neither should be trusted.