Since 2007 CNBC has published their ranking of states for doing business. The 2022 list is out and there’s some good news for California. The state moved up one slot in the rankings, from 33 to 32. But underlying that move are some curious changes. Stay tuned for a tale of inconsistency with CNBC chasing headlines and clicks.
As always, my methods are transparent. Click here to download my Excel workbook. All data is also available on the CNBC website. To see the individual state profile, click on the link to that state in the CNBC data table.
Introduction
CNBC ranks each state on 10 different criteria:
- Workforce
- Infrastructure
- Cost of Doing Business
- Economy
- Life, Health, & Inclusion
- Technology & Innovation
- Business Friendliness
- Education
- Access to Capital
- Cost of Living
Some of these criteria can be measured. The cost of living and access to capital are well-defined. Life, health, and inclusion is about as subjective as you can get. As we will see, CNBC’s main objective seems to be to generate clicks. They do this by wildly altering their scores and rankings within individual categories from year to year. Here I’ll focus on California because I live there.
Example: Education
CNBC is at least transparent about the high-level descriptions of their methodology. Here are the 2021 and 2022 descriptions of the factors considered in ranking education. As a preview I’ll note that California ranked 1 in education in 2021 and 11 in 2022. A change that large in a single year seems unlikely.
 2021 standards (click for larger image) |
 2022 standards (click for larger image) |
Pretty similar. Note that education covers K-12 plus higher education. California gets lots of credit for Stanford, Caltech, U.C. Berkeley, U.C. San Diego and a robust state university, college, and community college system. While that’s all well and good, I suspect they didn’t consider what’s going on in the classroom. Luckily we have a measurement of that.
Every three years the U.S. Department of Education issues “The Nation’s Report Card” better known as the National Assessment of Educational Progress (NAEP). https://nces.ed.gov/nationsreportcard/ This data is available by state and by major metropolitan area. The figures come from a series of eleven standardized tests covering civics, economics, geography, mathematics, music, reading, technology and engineering literacy, U.S. history, visual arts, vocabulary, and writing. In 2019 (the most recent survey year) California eighth graders ranked 40th in mathematics (tied with Oklahoma, South Carolina, and Rhode Island). In reading the state ranked 39th (tied with Arizona, South Carolina, and Arkansas). I don’t have time or energy to pursue the other nine subjects. But I will add that the data I looked at included charter and private schools. A quick look at math scores in public schools only shows the same results as all schools: California in a four-way tie for 40th place.
The fact is that Massachusetts ranks number 1 in math in all the NAEP surveys I’ve looked at. Ranking California first in 2021 is laughable.
CNBC Wants Clicks
The evidence is pretty compelling (at least for California). Let’s compare the rankings for 2021 and 2022:
California 2021 and 2022 (click for larger image)
The seven categories in red have rankings that changed a lot between the two years. I will say 2022 may be a correction of the 2021 results. Does anyone really believe that California ranked number 1 in Business Friendliness AND Education in 2021? For the latter, even number 11 in 2022 seems way too high. At least they figured out the state is not very business friendly.
And look at Technology and Innovation. Last in 2021 and first in 2022? The 2021 ranking is clearly wrong. For 2022, number 1 is debatable. Again, Massachusetts, Texas, Oregon, and probably a few other states are serious contenders in technology.
But there’s one conclusion we can draw. When seven of the ten categories’ rankings change that much from year to year, there is no consistency in the methodology. So why does CNBC do this?
Clicks and headlines are the short answer. When a state’s ranking changes a lot in either direction, residents of that state will notice. It makes little difference whether they gripe or lavish praise. Attention creates headlines and clicks.
Let’s take a closer look at California in 2021 and 2022.
The State of California
Here are the details supporting California’s ranking in both years.
 California 2021 (click for larger image) |
 California 2022 (click for larger image) |
For 2022, the rankings circled in red are the most egregious examples of bias. I present this data merely for your consideration with no further discussion.
Conclusion
If it sounds too good to be true, be very skeptical. CNBC’s “results” are trash. Readers are advised to ignore them.
