Machine Learning · Lesson 2 of 5
Decision Trees
~16 min
The Concept
Before building a forest, you need to understand one tree. Your supervisor draws a flowchart on the whiteboard: 'If slope is less than 4%, go left. If upstream drainage area is greater than 15 km², go left again. Now, did it flood?' The flowchart is a decision tree, the simplest machine learning model and also the most interpretable.
You trace your finger through the chart for a specific parcel, the one at the base of the agricultural slope near monitoring well GW-14. Slope 2.1%, drainage area 23 km², soil drainage index 3.2. It lands in the 'flooded' leaf. The field record confirms it flooded in three of the last five major events.
A decision tree is a sequence of binary yes/no questions about the input variables. At each step it asks one question about one variable, splits the data into two groups, and then asks another question about the best variable in each group, continuing until the groups are pure enough to make a confident prediction.
The tree 'learns' by finding the question at each step that best separates flooded from non-flooded parcels. The measurement it uses for 'best' is called Gini impurity (a score for how mixed-up a group is, zero means everyone in the group has the same answer), a number that is zero when a group is perfectly pure (all one outcome) and highest when the group is exactly half-and-half.
The Analogy
A decision tree plays a game like Twenty Questions. Is it bigger than a bread box? Does it have fur? Each yes/no question narrows down the possibilities until you can confidently guess the answer. The tree just picks the most useful question to ask at each step, the one that splits the remaining possibilities most cleanly.
The Math (optional)
Gini impurity measures how mixed a group is. If a node contains parcels where the proportion that flooded is p, Gini impurity is:
Gini impurity
G is 0 when all examples have the same label (p=0 or p=1) and 0.5 when they're exactly split (p=0.5). The tree always picks the split that minimises G in the resulting child nodes.
Weighted Gini after a split
nL and nR are the number of samples going left and right. The split is weighted by group size so a split that creates one tiny pure group and one huge mixed group isn't rated highly.
Try It
Click 'Grow tree' to add T1. Explore the splits the single tree makes, note which feature it uses at the root.
Feature importance across 1 tree
Each tree trains on a bootstrap sample of 120 Bluewater Basin sites. The forest votes: a site is predicted flooded if the majority of trees say so. Feature importance shows which variables drive the most splits across all trees.
Code (optional)
from sklearn.tree import DecisionTreeClassifier, export_text
import pandas as pd
# Load Bluewater Basin flood survey data
df = pd.read_csv("bluewater_flood_survey.csv")
features = ["slope_pct", "upstream_area_km2", "soil_drainage", "dist_to_river_m"]
X = df[features]
y = df["flooded"] # True/False
# Fit a single decision tree, max depth 4
tree = DecisionTreeClassifier(max_depth=4, random_state=42)
tree.fit(X, y)
# Human-readable text representation of the tree
print(export_text(tree, feature_names=features))
print(f"Training accuracy: {tree.score(X, y):.3f}")Line by line
- We load the flood survey: each row is one land parcel with four terrain features and a flooded label.
- DecisionTreeClassifier with max_depth=4 builds a tree at most 4 splits deep, deep enough to capture real patterns but shallow enough to inspect.
- export_text() prints the full tree as a human-readable if/else structure, you can trace any parcel through it by hand.
- tree.score() returns the proportion of training examples predicted correctly, but beware: a tree that's memorised the training data would score 1.0 without being useful for new data.
Why Real Researchers Care
Decision trees are used directly in environmental management when interpretability matters as much as accuracy, a flood risk model presented to a local council needs to be explainable in plain English, not a black box. The same split logic you're seeing here drives operational flood-warning systems in multiple countries.
Quick Check
Q1. What is Gini impurity measuring?
Q2. Why does a very deep tree risk being misleading?
Your Goal
Using the simulation, note the exact split values for the root node (first split) of T1. Could you apply that same rule by hand to a new parcel description? Try it.
Hint: Look at the feature name and threshold shown at the top of the tree diagram.
Teach It Back
Explain Gini impurity to someone without maths background, using a bag of coloured marbles as your analogy.