Regression & Prediction · Lesson 3 of 7

Multiple Regression

~15 min

The Concept

Rainfall explains 72% of streamflow variance, but 28% is still unexplained. Your supervisor suggests adding more predictors: temperature (which affects evaporation), soil moisture from the previous week, and land cover fraction. 'Each new predictor should earn its place,' she says. 'Adding variables always increases R², that doesn't mean you've improved the model.'

Multiple regression extends the simple case to multiple predictors simultaneously. Instead of one slope, there are several, each representing the change in the response associated with a one-unit change in that predictor, holding all others constant.

Adding predictors always increases R², even if those predictors are random noise. Adjusted R² corrects for this by penalising model complexity. If adding a predictor doesn't improve the model enough to offset the penalty for its added complexity, adjusted R² decreases.

The Analogy

It is like adding more ingredients to a soup hoping it gets better. Adding a good ingredient improves the flavor. Adding a random handful of something pointless might not make it worse, but it will not make it better either, and now you have a more complicated recipe for no real benefit. Adjusted R² is the taste test that only rewards ingredients that actually helped.

The Math (optional)

The multiple regression model generalises the simple case to p predictors.

Multiple linear model

y = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \cdots + \beta_p x_p + \varepsilon

Each predictor xⱼ has its own coefficient βⱼ, the partial effect of xⱼ on y when all other predictors are held constant.

Adjusted R²

R^2_{adj} = 1 - (1-R^2) \frac{n-1}{n-p-1}

n is sample size; p is the number of predictors. Adjusted R² decreases when adding weak predictors, it only increases when a new predictor genuinely improves fit relative to its cost.

Code (optional)

multiple_regression.pypython

import numpy as np
import pandas as pd
from sklearn.linear_model import LinearRegression

df = pd.read_csv("bluewater_monthly_features.csv")
# Columns: rainfall, temperature, soil_moisture, ag_fraction, streamflow

X = df[["rainfall", "temperature", "soil_moisture", "ag_fraction"]]
y = df["streamflow"]

model = LinearRegression().fit(X, y)
r2      = model.score(X, y)
n, p    = len(y), X.shape[1]
r2_adj  = 1 - (1 - r2) * (n - 1) / (n - p - 1)

for name, coef in zip(X.columns, model.coef_):
    print(f"  {name:20s}: {coef:+.4f}")
print(f"R²: {r2:.3f}   Adj-R²: {r2_adj:.3f}")

Line by line

X contains all four predictors simultaneously, each gets its own coefficient.
model.coef_ gives partial slopes, e.g., the rainfall coefficient now means 'effect of rainfall, holding temperature, soil moisture, and ag_fraction constant'.
Comparing R² and Adj-R² reveals whether the added predictors genuinely help or just inflate the apparent fit.

Why Real Researchers Care

Multiple regression is the workhorse of environmental modelling. Rainfall-runoff models, groundwater recharge estimates, and water quality predictions in published literature almost universally use multiple predictors. The key discipline is always the same: adjusted R², cross-validation, and residual checks, not raw R².

Quick Check

Q1. You add a fifth predictor to a model and R² increases from 0.73 to 0.741, but adjusted R² drops from 0.718 to 0.712. What does this mean?

Your Goal

Your multiple regression has R² = 0.83 and Adj-R² = 0.79 with 4 predictors. You add a 5th (atmospheric pressure) and R² becomes 0.834, Adj-R² = 0.787. Should you keep atmospheric pressure? Why or why not?

Hint: If Adj-R² decreases, the predictor is costing more in model complexity than it's contributing in fit.

Teach It Back

Explain why R² always increases when you add a predictor, even a useless one, and why adjusted R² was invented.

Previous lesson