Thanks to Iowahawk for this example. Let’s begin by defining Simpson’s Paradox. Then we can see how it applies to U.S. education.
Simpson’s Paradox
From the Stanford Encyclopedia of Philosophy:
Simpson’s Paradox is a statistical phenomenon where an association between two variables in a population emerges, disappears or reverses when the population is divided into subpopulations. For instance, two variables may be positively associated in a population, but be independent or even negatively associated in all subpopulations. Cases exhibiting the paradox are unproblematic from the perspective of mathematics and probability theory, but nevertheless strike many people as surprising. Additionally, the paradox has implications for a range of areas that rely on probabilities, including decision theory, causal inference, and evolutionary biology. Finally, there are many instances of the paradox, including in epidemiology and in studies of discrimination, where understanding the paradox is essential for drawing the correct conclusions from the data.
The article continues with a discussion of the mathematics of this paradox. The math is mostly set theory and statistical models, so it’s pretty accessible.
Application to Education
I’ve written about the National Assessment of Educational Progress (NAEP) before. Often called The Nation’s Report Card, the assessment tests students in grades 4, 8, and 12. The data tends to be more robust for grade 8. The survey is done every two years with one notable exception: the 2019 appraisal was followed by 2022. The extra year interval was undoubtedly an attempt to bypass 2020 data. That’s the COVID year. I’ve found it works pretty well to just omit those 12 months from datasets. Naturally, there are ongoing effects in education. I will look this problem squarely in the eye, shrug, and move on.
Here’s what Iowahawk has to say about Simpson’s Paradox and U.S. education.
