Quantifying Uncertainty · Lesson 3 of 6
Bootstrap Ensembles
~15 min
The Concept
You don't have a closed-form formula handy for the uncertainty of your streamflow regression's predictions at every rainfall value. Your supervisor suggests a trick that works almost regardless of the model: resample your data with replacement, refit, and repeat, many times.
Bootstrapping (a trick where you repeatedly re-draw your own data with replacement to see how much your results would wobble) simulates 'what if I'd collected a slightly different dataset by chance?' by drawing new samples, with replacement, from your existing data and refitting your model each time. The spread across all those refits, the ensemble, is a direct, empirical picture of your uncertainty, without needing any assumption about the shape of that uncertainty.
The Analogy
Imagine you have a bag of 20 marbles and want to know how much the average color mix might wobble by chance. You reach in, grab a handful, note the mix, put them back, and grab again, over and over. The spread across all those handfuls tells you how much to trust any single handful as a picture of the whole bag.
Try It
Watch the prediction band emerge from an ensemble of bootstrap-resampled regression fits. Increase the ensemble size and see the band stabilize.
Each thin teal line is one regression fit to a bootstrap resample of the same 22 data points — resampling with replacement simulates "what if we'd collected slightly different data by chance?" The gold band is the spread of predictions across the ensemble at each rainfall value: your honest uncertainty, not a single overconfident line.
Code (optional)
import numpy as np
rainfall = np.array([...]) # from Mission 06's dataset
streamflow = np.array([...])
n_boot = 500
predictions_at_100mm = []
rng = np.random.default_rng(seed=42)
for _ in range(n_boot):
idx = rng.integers(0, len(rainfall), size=len(rainfall))
x_sample, y_sample = rainfall[idx], streamflow[idx]
slope, intercept = np.polyfit(x_sample, y_sample, deg=1)
predictions_at_100mm.append(slope * 100 + intercept)
lower, upper = np.percentile(predictions_at_100mm, [2.5, 97.5])
print(f"95% bootstrap interval at 100mm rainfall: [{lower:.2f}, {upper:.2f}] m3/s")Line by line
- rng.integers draws random row indices with replacement, the core of bootstrap resampling.
- We refit a simple linear regression on each resample, exactly as in Mission 06.
- Collecting the prediction at one specific rainfall value (100mm) across 500 resamples gives us an empirical distribution of plausible predictions.
- The 2.5th and 97.5th percentiles of that distribution form a 95% bootstrap interval, no formula required.
Why Real Researchers Care
Bootstrapping is used across environmental science whenever the underlying uncertainty formula is unavailable, intractable, or the model violates the assumptions a formula would require. It's one of the most broadly applicable uncertainty quantification techniques in a researcher's toolkit.
Quick Check
Q1. What does resampling 'with replacement' mean in bootstrapping?
Your Goal
In the simulation, increase the ensemble size from 5 to 60. Does the prediction band get wider or narrower, and why does that make sense?
Hint: More resamples make the estimate of the band's edges more stable, it's not about the true uncertainty growing.
Teach It Back
Explain bootstrapping to a colleague as if you were describing it as a physical process with index cards, not code.