Regression & Prediction · Lesson 5 of 7
Generalised Additive Models
~18 min
The Concept
The residuals from the multiple regression show a mild curve, the relationship between rainfall and streamflow isn't quite linear at the extremes. Very low rainfall produces almost no streamflow regardless of the exact amount; very high rainfall produces disproportionately large events. A straight line can't capture this.
Your supervisor introduces Generalised Additive Models (GAMs): a flexible extension of regression where each predictor gets its own smooth curve, fitted to the data. The curves are added together, just like in multiple regression, but they don't have to be straight.
A GAM (a flexible regression that fits gentle curves instead of forcing everything into straight lines) replaces each linear term β · x with a smooth function s(x), a flexible curve that can bend to follow the data. The curves are estimated from the data itself, not assumed in advance. This makes GAMs much more flexible than linear regression without requiring you to guess the exact functional form.
The smoothness of each curve is controlled by a penalty, too flexible and you overfit; too rigid and you miss real patterns. GAMs automatically balance this trade-off during fitting.
The Analogy
Regular regression is like being told you can only connect the dots with one straight ruler. A GAM hands you a flexible piece of wire instead, one that can bend gently to hug the actual shape of the dots, as long as it does not wiggle so wildly that it just connects every single dot exactly.
The Math (optional)
A GAM replaces the linear predictor with a sum of smooth functions.
GAM formulation
Each s_j(x_j) is a smooth spline function fitted to the data. In a linear model, each s_j would be forced to be straight (βⱼ·xⱼ). GAMs remove that constraint while still keeping the additive structure.
Code (optional)
from pygam import LinearGAM, s
import numpy as np
import pandas as pd
df = pd.read_csv("bluewater_monthly_features.csv")
X = df[["rainfall", "temperature", "soil_moisture"]].values
y = df["streamflow"].values
gam = LinearGAM(s(0) + s(1) + s(2)).fit(X, y)
print(f"GAM R²: {gam.statistics_['pseudo_r2']['McFadden']:.3f}")
print(f"AIC: {gam.statistics_['AIC']:.2f}")
# Partial effects, one smooth curve per predictor
import matplotlib.pyplot as plt
fig, axes = plt.subplots(1, 3, figsize=(12, 4))
titles = ["Rainfall", "Temperature", "Soil Moisture"]
for i, ax in enumerate(axes):
XX = gam.generate_X_grid(term=i)
ax.plot(XX[:, i], gam.partial_dependence(term=i, X=XX))
ax.set_title(titles[i])
plt.savefig("gam_partials.png")Line by line
- s(0), s(1), s(2) tell the GAM to fit a smooth spline to each of the three columns in X.
- gam.fit() automatically selects smoothness penalties using cross-validation.
- partial_dependence() plots the estimated smooth curve for each predictor, this is what a GAM shows you that linear regression cannot.
Why Real Researchers Care
GAMs are widely used in ecology and environmental science because ecological and hydrological relationships are rarely linear. Species distribution models, pollution-response curves, and streamflow-rainfall relationships in complex basins are all commonly fitted with GAMs in peer-reviewed literature. The mgcv package in R, which implements GAMs, has over 20,000 citations.
Quick Check
Q1. What advantage does a GAM have over multiple linear regression for the rainfall-streamflow relationship?
Your Goal
A GAM on the Bluewater data shows the rainfall smooth is roughly linear for rainfall < 50 mm/month but curves sharply upward above 100 mm. What does this tell you about how the watershed responds to low vs high rainfall?
Hint: Think about what happens to soil storage capacity as rainfall increases, at some point the basin 'saturates' and additional rain converts almost entirely to runoff.
Teach It Back
Explain the difference between linear regression and a GAM to someone who only knows regression, using a hand-drawn curve as your main illustration.