Scientific Statistics · Lesson 6 of 7
Non-Parametric Tests
~12 min
The Concept
Before running the t-test, your supervisor checks the distribution of nitrate measurements. She plots a histogram. It's not symmetric, it's right-skewed, with a long tail toward high values. 'The t-test assumes the data is approximately normal,' she says. 'With this distribution and this sample size, we should check whether a non-parametric alternative changes our conclusion.'
Parametric tests like the t-test assume the data comes from a distribution with a specific shape (usually normal). Non-parametric tests (tests that make no assumption about the shape of the data) make no such assumption, they rank the data instead of using the raw values, which makes them robust to skewness and outliers.
The Mann-Whitney U test is the non-parametric counterpart to the two-sample t-test. It asks: if you drew one measurement from Year 1 and one from Year 5 at random, how often is the Year 5 value higher? If the answer is significantly more than 50% of the time, there's evidence of an increase.
The Analogy
Instead of measuring exactly how much taller every kid in class five is compared to class one, imagine lining up one kid from each class and just asking, who is taller, over and over with random pairs. You do not need exact heights to notice a pattern, just who tends to win the comparison. That is the spirit of a non-parametric test.
Code (optional)
import numpy as np
from scipy import stats
year1 = np.load("gw14_nitrate_year1.npy")
year5 = np.load("gw14_nitrate_year5.npy")
# Mann-Whitney U test (non-parametric alternative to t-test)
stat, p = stats.mannwhitneyu(year5, year1, alternative="greater")
# Probability that a random Year 5 value exceeds a random Year 1 value
# This is the "common language effect size" (AUC)
auc = stat / (len(year1) * len(year5))
print(f"Mann-Whitney U: {stat:.1f}")
print(f"p-value: {p:.5f}")
print(f"P(Y5 > Y1): {auc:.3f}")Line by line
- mannwhitneyu computes the test statistic by ranking all measurements and comparing rank sums.
- alternative='greater' makes this a one-sided test matching our pre-registered hypothesis.
- auc (area under the ROC curve) is a highly interpretable effect size: 0.5 means no difference, 1.0 means Year 5 is always higher.
Why Real Researchers Care
Non-parametric tests are standard in environmental science precisely because environmental data is often right-skewed: pollutant concentrations, rainfall totals, and species counts all tend to have long right tails. The Mann-Whitney test is used by the US Geological Survey, Environment Canada, and most national monitoring agencies as a robust alternative to t-tests for trend detection in water quality data.
Quick Check
Q1. When should you prefer a non-parametric test over a t-test?
Your Goal
The t-test gave p = 0.031 and the Mann-Whitney gave p = 0.027 for the same GW-14 comparison. The two tests agree. When might they meaningfully disagree, and which would you trust more in that case?
Hint: Think about what happens when the data is heavily skewed, which test's assumptions are still valid?
Teach It Back
Explain to a colleague what the Mann-Whitney test actually compares (in plain language, without mentioning ranks), using the GW-14 nitrate data as your example.