Hypothesis Testing in Practice

The Concept

It's been five years since Mission 00 when you pre-registered your hypothesis: 'Nitrate at well GW-14 has increased beyond normal year-to-year variation.' You now have five complete years of data. It's time to actually test it.

Your supervisor reminds you of the commitment you made then: the test was specified before seeing Year 5 data. 'That's what makes the result meaningful,' she says. 'If you'd looked at the data first and then chosen which test to run, the p-value would be meaningless.'

A hypothesis test asks: if the null hypothesis were true (no real change), how often would we see data this extreme purely by chance? If that probability, the p-value (a number that says how surprising your data would be if nothing had really changed), is very small, we have evidence against the null. We don't prove the alternative; we just show the data is hard to explain if the null is true.

A two-sample t-test compares the means from two independent groups, in our case, Year 1 nitrate measurements versus Year 5 nitrate measurements, and asks whether the difference between the two sample means is larger than random variation could plausibly produce.

The Analogy

Imagine flipping a coin ten times and getting eight heads. Is the coin rigged, or did that just happen by chance? A hypothesis test is the formal version of that gut check, it calculates exactly how surprising eight heads would be from a perfectly fair coin, so you know whether to be suspicious.

The Math (optional)

The t-test statistic measures how many standard errors the observed difference is away from zero.

Two-sample t statistic

t = \frac{\bar{x}_1 - \bar{x}_2}{\sqrt{\frac{s_1^2}{n_1} + \frac{s_2^2}{n_2}}}

x̄₁ and x̄₂ are the two sample means; s₁, s₂ their standard deviations; n₁, n₂ their sizes. The larger |t|, the more the means diverge relative to their combined sampling uncertainty.

Code (optional)

hypothesis_test.pypython

import numpy as np
from scipy import stats

year1 = np.load("gw14_nitrate_year1.npy")
year5 = np.load("gw14_nitrate_year5.npy")

# One-sided t-test: has the mean increased?
# alternative='greater' means H1: mu_year5 > mu_year1
t_stat, p_value = stats.ttest_ind(year5, year1, alternative="greater")

print(f"Year 1 mean: {year1.mean():.3f} mg/L (n={len(year1)})")
print(f"Year 5 mean: {year5.mean():.3f} mg/L (n={len(year5)})")
print(f"t statistic: {t_stat:.4f}")
print(f"p-value:     {p_value:.5f}")
print(f"Evidence against H0: {'Yes' if p_value < 0.05 else 'No'}")

Line by line

We use a one-sided test (alternative='greater') because our pre-registered hypothesis was directional: an increase, not just any change.
stats.ttest_ind computes the Welch t-test, which doesn't assume equal variances, appropriate here.
The p-value is the probability of seeing a t-statistic this large (or larger) if H0 were true.
We compare to α = 0.05, the threshold we pre-registered in Mission 00.

Why Real Researchers Care

The pre-registration you completed in Mission 00 is a real scientific practice used by major environmental monitoring bodies. The US EPA and many state agencies require pre-specified statistical tests in their monitoring protocols precisely to prevent p-hacking. Your p-value from this test is trustworthy because the test was committed to before the data arrived.

Quick Check

Q1. Your p-value is 0.031, and your pre-registered α was 0.05. What do you conclude?

Q2. Why does using a one-sided versus two-sided test matter?

Your Goal

Compare your actual Year 5 hypothesis test result with the hypothesis you pre-registered back in Mission 00 Lesson 4. Did the test confirm, contradict, or complicate your pre-registered prediction?

Hint: Pull up your Mission 00 Lesson 4 teach-back notes. The question isn't whether you were 'right', it's whether you followed through on what you committed to.

Teach It Back

Explain to a non-scientist what a p-value actually means, without using the phrase 'probability the null hypothesis is true'.