Lesson 7 of 9 · about 17 minutes
Correlation, Causation and the Traps Between Them
You'll be able to
- Explain what a correlation coefficient measures and what it cannot show
- Name a likely confounder for a claimed cause and effect
- Spot regression to the mean in before-and-after claims
- Work a Simpson's paradox table and explain the reversal
- Write causal claims only as strongly as the evidence allows
Two numbers move together. Someone wants a headline that says one caused the other. Your job is to slow down long enough to ask what else could be going on, because the answer is usually "plenty."
What a correlation measures
The correlation coefficient, r, runs from minus 1 to 1. Near 1, the two variables rise together in a fairly straight line. Near minus 1, one rises as the other falls. Near 0, there is no straight-line pattern. Spreadsheets compute it with =CORREL(A2:A7,B2:B7).
Example: fire engines and fire damage
Six invented fires in Harrow County:
| Fire | Engines dispatched | Damage ($ thousands) |
|---|---|---|
| 1 | 2 | 41 |
| 2 | 3 | 44 |
| 3 | 5 | 47 |
| 4 | 6 | 52 |
| 5 | 8 | 55 |
| 6 | 10 | 63 |
CORREL returns 0.99. Square it and r squared is 0.98: in this little dataset, engine count lines up with almost all the variation in damage. Should the county send fewer engines?
Obviously not. Big fires get more engines, and big fires do more damage. The size of the fire is the confounder, and it drives both columns.
The fire example is silly on purpose, so the logic is easy to see. Real ones are harder. Towns with more parks report less obesity. Schools with more counselors have higher graduation rates. Counties with more broadband have more new businesses. Each might be partly causal. Each also has an obvious candidate confounder: money. Wealthier places tend to have more parks, more counselors and more broadband, and better outcomes on many measures for many reasons.
Finding the confounder
When someone claims X causes Y, ask three questions, in this order:
- Could Y cause X instead? (Do struggling schools get extra counselors because they are struggling?)
- What third thing could cause both? Income, age, population size, season and geography are the usual suspects.
- How were the groups formed? If people chose the program themselves, the people who sign up may differ from those who don't.
The strongest evidence for causation comes from a randomized controlled trial, where a coin flip decides who gets the treatment, so confounders spread evenly across both groups. Most local data isn't that. It is observational: someone recorded what happened. Observational data can still support causal conclusions, with care, comparison groups and several lines of evidence. But the bar is higher, and your verbs should show it.
| Evidence you have | Verbs that fit |
|---|---|
| Two numbers move together | linked, associated, coincided with |
| Before-and-after with a fair comparison group | followed, was accompanied by a larger drop than |
| Randomized trial or strong natural experiment | caused, reduced, increased |
Regression to the mean
Every measurement mixes a steady part with luck. Pick things because they had an extreme year, and much of that extreme was luck. Next year the luck evens out and they drift back toward normal, even if nobody did anything.
Example: the camera that took credit
Invented Brannock County ranks its intersections by 2024 crashes and installs cameras at the ten worst on January 1, 2025. Across all county intersections the average was 6 crashes a year.
| Group | 2024 average crashes | 2025 average crashes | Change |
|---|---|---|---|
| Ten worst, cameras installed | 14 | 9 | (9 minus 14) / 14 = minus 35.7% |
| Next ten worst, no cameras | 12 | 8.5 | (8.5 minus 12) / 12 = minus 29.2% |
The press release says cameras cut crashes 36 percent. But the intersections without cameras fell 29 percent too, because both groups were picked for a bad year. The camera effect, if any, looks closer to the gap between the groups, a few percentage points, and even that needs more years of data to trust.
Regression to the mean shows up anywhere selection follows an extreme: the worst schools getting a new principal, the county with a record overdose year starting a program, the team that fires its coach after a losing season. The fix is a comparison group and several years of data before the change.
Simpson's paradox
This one feels like a magic trick. A pattern holds in every subgroup, then flips when you add the groups together.
Example: two invented hospitals
| Hospital | Case type | Patients | Survived | Survival rate |
|---|---|---|---|---|
| Regional | Serious | 400 | 320 | 80.0% |
| Regional | Minor | 100 | 98 | 98.0% |
| Regional | All | 500 | 418 | 83.6% |
| Community | Serious | 100 | 75 | 75.0% |
| Community | Minor | 400 | 380 | 95.0% |
| Community | All | 500 | 455 | 91.0% |
Regional does better with serious cases (80 versus 75 percent) and with minor cases (98 versus 95 percent). Yet overall, Community wins, 91.0 percent to 83.6 percent.
The arithmetic: Regional's 500 patients are 80 percent serious cases (400 of 500). Community's are 20 percent serious (100 of 500). Serious cases have lower survival everywhere. The overall rate mostly measures the case mix. A headline that says "Community Hospital patients more likely to survive" is accurate and points readers to the wrong hospital.
The most famous real case is the 1973 graduate admissions data at the University of California, Berkeley. Of 8,442 men who applied, about 44 percent were admitted; of 4,321 women, about 35 percent. Statisticians Peter Bickel, Eugene Hammel and J. W. O'Connell looked department by department in a 1975 paper in Science. They found women had applied in larger shares to departments that were hard to get into. Department by department there was little sign of bias against women, and after accounting for where people applied, the data showed a small bias in their favor.
The lesson for a reporter: when you compare groups, check whether their makeup differs on something that drives the outcome. Then compare within subgroups. Ages, case severity, income levels and school grade levels are common culprits.
Common mix-ups
- Claim: A strong correlation proves causation. What the evidence shows: r measures how closely two numbers move together. It says nothing about why.
- Claim: If crashes fell after the new program, the program worked. What the evidence shows: Without a comparison group, you can't separate the program from regression to the mean and general trends.
- Claim: Combined totals are more reliable than subgroups. What the evidence shows: Combining can create a reversal when groups differ in mix. Look at both.
Reporting angle
- When an agency credits a program for an improvement, ask for several years of data before it began and for a comparison group of similar places or people without it.
- Ask how sites or participants were chosen. "The worst ones" is a red flag for regression to the mean.
- For hospital, school or jail comparisons, ask for results broken out by severity, grade level or risk level.
- Ask a statistician at a nearby university to look at your comparison before publication. Many will do it for free if you ask a specific question.
Try it
- Enter the fire table in a spreadsheet and confirm
=CORRELreturns about 0.99. - Build the hospital table with formulas for each survival rate. Change Regional's serious cases to 100 patients with 80 survivors and its minor cases to 400 with 392 survivors. What happens to the overall comparison? (Regional should now lead overall, 94.4 to 91.0 percent.)
- Find a recent local claim that a policy "caused" an improvement. Write down the likely confounder, whether regression to the mean could apply, and what comparison you would ask for.
Check yourself
1. Fires with more engines dispatched have more damage (r = 0.99). What is the most likely explanation?
A third factor, the size of the fire, drives both. Strong correlation alone cannot tell you which way, or whether, causation runs.
2. Cameras went in at the 10 worst intersections after a bad year, and crashes there fell 36 percent. The next 10 worst, with no cameras, fell 29 percent. What is the main lesson?
Sites picked for an extreme year tend to drift back toward normal anyway. The comparison group shows most of the drop would have happened without cameras.
3. In the hospital table, Regional has higher survival for serious cases (80 vs 75 percent) and minor cases (98 vs 95 percent), yet lower overall (83.6 vs 91.0 percent). Why?
Regional's patients are 80 percent serious cases; Community's are 20 percent. The mix of cases, not the quality of care, drives the overall gap.
4. In the 1973 Berkeley graduate admissions data, men were admitted at about 44 percent and women at about 35 percent. What did Bickel, Hammel and O'Connell find when they looked department by department?
The paper found the proportion of women applicants was high in departments with low admission rates. Accounting for that, the aggregate showed a small bias in favor of women.
5. Which wording best fits an observational study finding that towns with more parks report less obesity?
Observational data shows association. Wealthier towns may have more parks and lower obesity for many reasons. Say 'linked' or 'associated,' and name plausible confounders.
Terms in this lesson
- Correlation coefficient (r)
- A number from minus 1 to 1 measuring how closely two variables follow a straight-line relationship.
- Confounder
- A third factor that affects both the supposed cause and the supposed effect, creating a misleading link.
- Regression to the mean
- The tendency of unusually high or low measurements to be followed by ones closer to average, with no intervention at all.
- Simpson's paradox
- A pattern that appears in every subgroup but reverses when the groups are combined, because the groups differ in size or mix.
- Randomized controlled trial
- A study that assigns the treatment by chance, so confounders are spread evenly between groups.