Regression & Prediction · Lesson 1 of 7

What Regression Actually Does

~14 min

The Concept

A hydrological engineer wants to predict streamflow at the Bluewater River gauge from rainfall, not because she can't measure streamflow directly, but because rainfall forecasts are available 72 hours ahead and streamflow sensors sometimes fail. A predictive model would let the team anticipate flood conditions before they arrive.

Linear regression is the first tool she reaches for. Not because it's always the right answer, it often isn't, but because it's transparent, interpretable, and its failures are easy to diagnose.

Regression finds the mathematical relationship between one or more input variables (predictors) and an output variable (the response). The simplest version fits a straight line: for every 1-unit increase in rainfall, streamflow increases by some amount. That 'some amount' is the slope, the thing regression estimates.

Ordinary least squares (OLS, a method for drawing the single best straight line through a cloud of dots) regression finds the line that minimises the sum of squared vertical distances from each data point to the line. Those distances are residuals (the leftover gaps between what actually happened and what the line predicted). The line that makes residuals as small as possible (in a squared sense) is the best linear fit.

The Analogy

Imagine a bunch of kids standing at slightly different heights, and you want to draw one straight ramp that comes as close as possible to touching every kid's head. Regression is the process of finding exactly where to place that ramp so the total distance between the ramp and every head is as small as it can possibly be.

The Math (optional)

OLS minimises the sum of squared residuals to find the best-fit slope (β₁) and intercept (β₀).

Simple linear model

y = \beta_0 + \beta_1 x + \varepsilon

y is the response (streamflow), x is the predictor (rainfall), β₀ is the intercept, β₁ is the slope, and ε is the irreducible error, the part no model can explain.

OLS slope

\hat{\beta}_1 = \frac{\sum(x_i - \bar{x})(y_i - \bar{y})}{\sum(x_i - \bar{x})^2}

The covariance of x and y divided by the variance of x. This captures how much y tends to change when x changes, relative to how much x itself varies.

Try It

Drag the slope slider away from the OLS best fit and watch R² and residuals respond, proving visually why the OLS line is optimal.

Slope override0.159 (best fit)

Show residuals

slope = 0.159intercept = 4.15R² = 0.772

Drag the slope away from "best fit" and watch R² drop — the ordinary least squares line specifically minimizes total squared residual distance, which is why it maximizes R² for a straight-line fit.

Code (optional)

simple_regression.pypython

import numpy as np
from sklearn.linear_model import LinearRegression
import matplotlib.pyplot as plt

rainfall   = np.load("bluewater_monthly_rainfall.npy").reshape(-1, 1)
streamflow = np.load("bluewater_monthly_streamflow.npy")

model = LinearRegression().fit(rainfall, streamflow)
print(f"Intercept: {model.intercept_:.3f} m³/s")
print(f"Slope:     {model.coef_[0]:.4f} m³/s per mm of rainfall")
print(f"R²:        {model.score(rainfall, streamflow):.3f}")

Line by line

reshape(-1, 1) converts the 1D array into a column vector, sklearn expects 2D input for predictors.
model.coef_[0] is β̂₁: the estimated increase in streamflow (m³/s) per additional mm of rainfall.
model.score() returns R², the fraction of variance in streamflow explained by the linear rainfall relationship.

Why Real Researchers Care

Rating curves, empirical relationships between water level and streamflow, are one of the oldest and most widely used applications of regression in hydrology. Every USGS and Environment Agency streamflow estimate you'll ever see was derived using regression between a measured property (stage height) and an unmeasured one (discharge). You're building the same type of model.

Quick Check

Q1. What does OLS regression minimise?

Your Goal

Using the regression simulation above: find the slope value that achieves the highest R². Now drag the slope to 0.05 units above the OLS value. By how much does R² drop? What does this tell you about the sensitivity of R² near its maximum?

Hint: R² is relatively flat near the optimum, large slope changes are needed to cause dramatic R² drops.

Teach It Back

Explain to someone who has never seen regression what a residual is, using the Bluewater rainfall-streamflow example, without using the word 'regression'.