Relationships

Simple Linear Regression

Fit a straight line that predicts one variable from another.

Advertisement

Theory

Simple linear regression estimates an intercept and slope for the line ŷ = a + bx. The slope is the expected change in y for a one-unit increase in x.

ŷ = a + bx

What you will learn

  • Estimate slope and intercept.
  • Use a fitted line for prediction.
  • Interpret residuals and model limitations.

How to interpret it

The slope estimates the average change in predicted y for a one-unit increase in x. The intercept is the predicted y when x equals zero, but it may have no practical meaning when zero lies outside the observed range.

Assumptions and limitations

Simple least-squares inference typically assumes a linear relationship, independent observations, roughly constant residual spread, and approximately normal residuals for tests and intervals. Predictions are most reliable within the observed x range.

Common mistakes

  • Extrapolating far beyond the data.
  • Treating the fitted line as proof of causation.
  • Ignoring residual patterns that reveal nonlinearity or unequal variance.
Try it yourself

Problem question

For points (1,2), (2,4), and (3,5), estimate the regression line.

Step-by-step solution

  1. Compute x̄ = 2 and ȳ = 3.667.
  2. The slope is Σ(x−x̄)(y−ȳ) / Σ(x−x̄)² = 1.5.
  3. The intercept is 3.667 − 1.5(2) = 0.667.
  4. The fitted line is ŷ = 0.667 + 1.5x.
Calculator results are educational estimates. Confirm important academic or professional work independently.