Physics-Informed AI · Lesson 3 of 6
Fitting a Flexible Model
~12 min
The Concept
You build the naive version first, a flexible curve fit purely to the eight points, with no physics attached, exactly the one that got you in trouble in Lesson 1. Your supervisor wants you to see it fail clearly before you fix it.
A sufficiently flexible model (like a high-degree polynomial or an unconstrained neural network) will bend to match noise as eagerly as it matches signal. With eight points and enough flexibility, it can produce a curve that technically has the lowest possible error on your data, and is still nonsense physically.
The Analogy
It is like a tailor who is told to make a suit fit eight specific measurements perfectly, with no other guidance. They might sew something that hits every number exactly but looks like nothing a human could actually wear. Getting every point right does not automatically mean the result makes sense.
Code (optional)
import numpy as np
from numpy.polynomial import polynomial as P
distance = np.array([2, 8, 15, 25, 40, 60, 85, 110])
head = np.array([38.1, 30.4, 27.9, 22.6, 20.1, 19.8, 21.3, 18.9]) # noisy field data
# A flexible 6th-degree polynomial will fit these 8 points almost exactly ,
# but has no idea that head "shouldn't" rise between distance=60 and 85.
coeffs = P.polyfit(distance, head, deg=6)
fitted = P.polyval(distance, coeffs)
print("Fitted values:", np.round(fitted, 2))
print("Max residual:", np.round(np.max(np.abs(fitted - head)), 3))Line by line
- We fit a degree-6 polynomial, deliberately more flexible than eight points can reliably support.
- polyval evaluates the fitted curve at our original distances to check the fit quality.
- The residuals will be tiny, which looks great by pure error metrics, but says nothing about whether the curve behaves physically between the measured points.
Why Real Researchers Care
This is a textbook case of overfitting, but with an environmental science twist: the danger isn't just poor prediction on new data, it's producing an answer that violates a law of physics the researcher already knows to be true. A policymaker acting on this curve could be told the water table is recovering when Darcy's Law says it can't be.
Quick Check
Q1. What's the specific danger of overfitting in this hydrogeology context, beyond ordinary prediction error?
Your Goal
Using the code above, would increasing the polynomial degree to 7 make the physical violation more or less likely between distance=60 and 85? Explain your reasoning without running new code.
Hint: More flexibility generally means more freedom to wiggle between points, not less.
Teach It Back
Explain overfitting to someone who has never heard the term, using the well-head example.