Lately, there has been some discussion about employment figures revisions. The most recent entry is from National Review’s Dominic Pino with the blaring headline, “No, the Biden Administration Is Not Manipulating Jobs Data.” Sadly, Mr. Pino misses the trees by focusing on the forest. In this case, the forest is the annual revision of the Bureau of Labor Statistics “CES [current employment statistics] Preliminary Benchmark Announcement.” For March, 2024, the revision was -818,000 (-0.5%). While the job loss figure looks big, total nonfarm employment in that month was 158,106,000. The US economy is really, really big. (And the actual percentage was -0.5174%.)
I can’t let this pass without noting Brian Wesbury’s observation about the Benchmark Revision:
(click for larger image)
By focusing on the annual revision, Mr. Pino misses the routine monthly revisions. Here’s the BLS explanation of those revisions.
The Current Employment Statistics (CES) first preliminary estimates of employment, hours, and earnings are published each month approximately 3 weeks after the reference period. Estimates are then revised twice, before being held constant until the annual benchmarking process. Second preliminary estimates for a given month are published the month following the initial release, and final sample-based estimates are published 2 months after the initial release. Technical information on revisions can be found in the revisions section of the CES handbook of methods webpage.
Executive Summary
Here I want to look at the monthly revisions. The issue is whether the third estimate minus the first estimate is positive (+), negative (-), or zero (0). A negative difference means a downward revision. Positive means upward. Since my concern is downward revisions, I included 0 values with the positives. My conclusions:
- First I looked at streaks. Changes and be positive or negative. Think of this as flipping a coin, a true random event. Heads (-) and tails (+) each have a 50% probability. Between January and June, 2023 there were six consecutive months with downward revisions (out of a total of 17 months between January, 2023 and May, 2024). This is equivalent to flipping a coin 17 times and getting a streak of six consecutive heads. While the math is forbidding, the probability of this happening by chance is between 5% and 10%. This is actually on the high side, as I only looked at the longest single streak. There are also one streak of four in a row and two streaks of two consecutive. That’s 14 downward revisions out of 17 total.
- Next, I used classical statistics, specifically the t-test. But there is an issue. If the number of upward and downward revisions were equally likely, we would expect 50% of each. But the data (January, 2010 – May, 2024 excluding calendar year 2020) shows 40.37% downward revisions. More on that difference later. I simply did the test twice, once with each population value. Using 40.37%, the t-statistic was -3.783. (Reminder: the sign of the t-stat is irrelevant.) Using 50%, the t was 2.916. That, too, shows a significant change between the 2010 – 2024 and 2023-2024 periods. In fact, the p-value for the smaller t-statistic is 0.0101.
I conclude that it is very unlikely that the revisions to employment between January, 2023 and May, 2024 occurred by chance. Something smells here.
Click here to download my Excel workbook. (Statistical material is on the “Data useful” tab starting in row 46.)
Analytical Details
Streak Analysis
So here are the numbers. I created the third column (Date) and the last column (+ or -). A + entry means the “3rd-1st” column was greater than or equal to zero. A – entry means that column was less than zero. In other words, + means a revision upward (or zero) and – means a downward revision.
Employment revisions, raw data (click for larger image)
Here’s the summary for three years. Each ends in May, 2024 and starts in January of each year.
Employment revisions (click for larger image)
Notice something interesting. As we progress through the three samples, the percentage of changes that were negative steadily increases: 48.78% to 65.52% to 82.35%. The January to June 2023 figures are especially problematic. Between January and June 2023, every single revision was downward.
I decided to investigate. What are the chances that with seventeen observations, six consecutive figures would be negative?
Turns out this is identical to flipping a coin. Heads are negative revisions and tails are positive. Searching for coin flips, I found an interesting website, Omnicalculator.com. The probability of exactly six consecutive heads (or tails) is 5.36%. Here are the approximate and exact results:
 Probability of six consecutive approximate |
 Probability of six consecutive exact |
For those who are curious here are the probabilities of getting exactly x heads in 17 flips. The chart is misleading; probabilities are on the vertical axis and in percent. The horizontal axis is number of consecutive flips.
Probability graph (see note in text, click for larger image)
On first examination, this seems wrong. After all, in 17 flips you’re almost guaranteed to get 1 heads. What the chart is really measuring is exactly x heads in 17 flips. Thus, the probability of getting one heads and 16 tails is 3%. This happened because I specified “exact.” If instead I specified “at least” one heads, the probability rises to almost 100%. And the probability of at least six consecutive rises to 10%. Honestly, this seems like a better metric for our problem.
 Probability of at least six consecutive approximate |
 Probability of at least one consecutive approximate |
Just to be complete, here’s the “at least” graph for x consecutive.
Probabilities are never proof. But look at the trend. The percentage of changes that were negative rises from 48.8% to 65.5%, settling at 82.4%. If you don’t smell a rat, I recommend visiting an ear, nose, and throat clinic.
Classical Statistics Approach
Before diving into this, here are the summary statistics:
(click for larger image)
The large sample (our “population”) included monthly data from January, 2010 to December, 2019 and January, 2021 to May, 2024. That’s 161 observations. I omitted 2020 because of COVID. Using the entire sample, the percentage – is 40.37%. To be on the safe side, I also tested using 50%, the expected number of – observations for a truly random process. For n=161, the sample standard deviation is 0.1362. We can use a standard t-test with this approximation (ignoring the degrees of freedom).
The numerator is the difference between the percentage – for 2023-2024 minus the same percentage for January, 2010 – May, 2024 (excluding calendar year 2020). The denominator is the standard deviation of the percentage for the larger sample (n=161).
That’s just to give you some idea of what the t-statistic is all about. Luckily, we have the exact statistic courtesy of OmniCalculator.com. Using 40.37%, the t-statistic for – observations is -3.783. Using 50% the t is 2.916. (Reminder: the sign of the t-stat is irrelevant.) That, too, is significant.
The p-values are 0.0016 (using 40.37%) and 0.0101 (50%). We can safely reject the null hypothesis that the percentage of – value observations between January, 2023 and May, 2024 was statistically the same as the population (large sample) percentages (40.37% or 50%). The numbers have most likely been fudged.
Here are the full statistical results.
(click for larger image)
