This is part 3 of my report on the state of U.S. education in 2019. Here I’ll describe how we stack up internationally. My data is from the Trends in International Mathematics and Science Study (TIMSS) administered every four years. While my focus will be on 2019, the year 2011 has some interesting material. For comparison purposes, I try to look at the 8^th grade results.

Part 1 looked at the most recent NAEP results.  Part 2 looked at those results with a focus on California.

Here are the exceptions. In 2011 the TIMSS was deliberately aligned with NAEP results, mainly in science and mathematics. After reviewing the methodology, I concluded that the data (especially from the U.S.) would be, well, misleading. I’ll rely on the joint report issued by TIMSS and NAEP instead. See “U.S. States in a Global Context: Results From the 2011 NAEP-TIMSS Linking Study.” NAEP and TIMSS U.S., U.S. Department of Education, NCES 2013-460.

Executive Summary

The U.S. did better than I expected. That may have been caused by my belief that U.S. education generally falls short of global standards. I’ve changed my mind a bit. My concern has now shifted to the countries that are, frankly, eating our lunch in math and science. Do you care more that we are ahead of France and New Zealand by 30 points in general science, or that Singapore is beating the U.S. by 86 points?

Technical Notes

Some technical notes are needed. TIMSS tests all have 1,000 points possible. The scores for each subject are normalized so the average across all countries is 500. That means we can’t compare the absolute scores across various subject matter tests. I’ve omitted countries that reported no data on a test. If nothing else, that makes the charts readable.

TIMSS 2019 Results

TIMSS fields covered include mathematics. general science, biology, chemistry, earth science, and physics. The data is in the Excel workbook; feel free to have at it. The “working” worksheet is a copy of the original data. I edited it to delete countries not reporting. Then I sorted the 2019 results from high to low average scored. The “graph(s)” worksheet is where you find the graphs. (You probably could have figured that out.)

Mathematics

The U.S. (score of 515) ranks 11^th out of 21 countries reporting results for 2019. Singapore is number 1 (616) followed by Taiwan (612). At least we finished above Portugal, Italy, Turkey, and France.

(click for larger image)

General Science

The U.S. (522) is once again 11^th out of 21 countries. Singapore (608) is first again. Once again, we finished above Portugal, Italy, Turkey, and France. Reminder: absolute scores cannot be compared across subject categories.

(click for larger image)

Biology

The U.S. (530) is 10^th out of 21 countries. Singapore (633) is first again. Yet again, we finished above Portugal, Italy, Turkey, and France. 

(click for larger image)

Chemistry

The U.S. (509) is 15^th out of 21 countries. Singapore (616) is first again. This time Portugal and Turkey finished ahead of us. We still were ahead of Italy and France.

I’ll add a personal note here. In the distant past I got a degree in chemical engineering. Chemistry is different from other subjects in requiring a vast quantity of memorization. Organic chemistry is notorious as the course that weeds out possible future chemists. Memorization takes time. A lot of time. Perhaps U.S. students simply don’t put in the long hours needed to learn this subject. (I was also fortunate to have two semesters of chemistry in high school.)

(click for larger image)

Earth Science

The U.S. (530) is 13^th out of 21 countries. Taiwan (579) is first, with Singapore third. This time, we finished above Italy, Turkey, and France. Portugal was two countries above us with an average of 531. Reminder: absolute scores cannot be compared across subject categories.

(click for larger image)

Physics

The U.S. (515) is 15^th out of 21 countries. Singapore (619) is first again. Once again, we finished above Portugal, Italy, and France. Turkey finished two slots above us with an average of 519.

(click for larger image)

TIMSS 2011 Science Results

In 2011, the NAEP coordinated with TIMSS to offer synchronized tests in math and science. I had hoped that the tests were similar enough that the raw data would be useful. For better or worse, that is not the case. After the conclusion, I’ll add an appendix describing the methodology used to create comparable data.

As noted earlier, the data is not available (at least I couldn’t find it). Therefore I’ll rely on the report linked earlier.

One small caveat: I use terms like “above average group” for convenience. Here’s the full description from the above report:

Figure 6-A. Benchmark-level results in TIMSS eighth-grade science for students with average scores higher than the TIMSS average, by jurisdiction: 2011

Mathematics Results

Let me quote from the report (p. 16):

Average scores for over two-thirds of the U.S. states and one-quarter of the 47 education systems were higher than the TIMSS average (figure 3-A). Massachusetts scored higher than the TIMSS average and 42 of the 47 participating education systems. Average scores for 10 states were not significantly different from the TIMSS average (figure 3-B). Alabama, although the lowest scoring state, scored higher than 19 education systems (figure 3-C). Only four education systems—Chinese Taipei-CHN, Hong Kong SAR, the Republic of Korea, and Singapore—had TIMSS scores that were higher than the scores for all 52 states.

I know the next question: what’s figure 3-A? You have only to ask.

(click for larger image)

An example will help. Again, from the report:

How to Read the Graphics

In New Jersey (figure 3-A), 16 percent of students scored at the Advanced benchmark, and 50 percent scored at or above the High benchmark.

The astute reader will notice that California is missing from Table 3-A. That’s because the Golden State is included in “Benchmark-level results in TIMSS eighth-grade mathematics for students with average scores not significantly different from the TIMSS average, by jurisdiction: 2011” group. Table 3-A was “Benchmark-level results in TIMSS eighth-grade mathematics for students with average scores higher than the TIMSS average, by jurisdiction: 2011.” Ah. California is not in the above-average group.

There are 49 entities in the above average group. The U.S. is 46^th. Another 16 are in the average group:

(click for larger image)

And 35 more are in the below average group:

(click for larger image)

Conveniently, that’s 100 entities total. California ranks 65^th. Enough said.

Science Results

Let me again quote from the report (p. 20):

Average scores for 90 percent of the states and 40 percent of the participating education systems were higher than the TIMSS average (figure 6-A). Massachusetts and Vermont scored higher than the TIMSS average, the High benchmark, and scores for 43 participating education systems. Average scores for Arizona and California were not significantly different from the TIMSS average (figure 6-B). The District of Columbia, although the lowest scoring state, scored higher than 14 education systems (figure 6-C). Singapore was the only education system that scored higher than all 52 states.

Once again, California does not appear in the above average group:

(click for larger image)

The U.S. ranked 46^th. Perhaps the Golden State will be in the average group:

(click for larger image)

Sure enough, but just barely. By my count, California ranks 71^st out of 100 entities. For completeness, here’s the below average group. I know I feel better when I realize that California finished above Mississippi, Alabama, and the Republic of Macedonia.

(click for larger image)

Conclusion

The 2011 survey was truly a massive effort, with many more entities surveyed. At least the U.S. is slightly above average, ranking 46^th in both math and science. California’s performance continues to be dismal. I invite readers to see how your state stacks up.

The 2019 subject matter surveys put the U.S. technically above average (remember, by design the average score for each subject is 500). But our ranking compared to other countries ranges from mediocre to pretty bad. In particular, the East Asian countries are, in fact, eating our lunch. We should be happy we’re letting so many of their students into graduate programs in science and engineering. If only we’d follow the excellent suggestion to give each foreign Ph.D. in math, science, and engineering a green card along with their degree. I would extend that to economics and finance, but I have an obvious bias.

Appendix: Methodology

The methodology used to harmonize U.S. state data with TIMSS data is described in great detail on pages 22-27 of the report mentioned earlier. Here are some excerpts.

Sampling Procedure

To evaluate various linking methodologies, multiple samples of students were assessed during the NAEP testing window (January–March) as well as the TIMSS testing window (April–June).

• Students assessed in NAEP mathematics or science during the 2011 NAEP testing window (2011 NAEP national sample).
• Students assessed during the 2011 NAEP testing window with NAEP-like braided booklets containing both NAEP and TIMSS test questions (braided booklet samples in 2011 NAEP testing window).
• Students in the United States assessed in TIMSS mathematics and science during the 2011 TIMSS testing window (2011 TIMSS U.S. national sample).
• Students in the United States assessed during the 2011 TIMSS testing window with TIMSS-like braided booklets containing both NAEP and TIMSS test questions (braided booklet sample in 2011 TIMSS testing window).

All NAEP and TIMSS 2011 mathematics and science test questions at grade 8 were included in the NAEP-like and TIMSS-like braided booklets.

Linking Methodologies

Again, an overview from the report:

The process by which NAEP results are reported on the TIMSS scale is referred to as statistical linking. Mislevy (1992) and Linn (1993) proposed a type of taxonomy in categorizing the linking methodologies into four forms—equating, calibration, projection, and moderation. Linking NAEP and TIMSS is an effort to link assessments based on different frameworks. It is clear that equating is not a feasible approach (see Kolen and Brennan (2004) for the assumptions required for equating). The other three linking methods—moderation, projection, and calibration— can be applied in linking NAEP and TIMSS.

Statistical moderation aligns score distributions such that scores on one assessment are adjusted to match certain characteristics of the score distribution on the other assessment. In this study, moderation linking was accomplished by adjusting NAEP scores so that the adjusted score distribution for the nation’s public school students who participated in 2011 NAEP had the same mean and variance as the score distribution for the public school students in the 2011 TIMSS U.S. national sample. This allowed NAEP results to be reported on the TIMSS scale.