[Prof. Ronald Inglehart was kind enough to read and comment on this article. He has correctly noted that data from cohort analysis must be handled differently from the procedures economists mostly use. I have removed the entire section discussing data. He also notes that Roberto Foa is now an assistant professor at the University of Melbourne. I have corrected the biographical sketch to reflect this new information. For the sake of the historical record, click here for a pdf version of my original article. You can click here to read Prof. Inglehart’s response to the new Mounk and Foa paper (forthcoming in the Journal of Democracy).]
The quoted text in the title is the headline in a New York Times article November 29. The article reports on a paper that will be published in the January, 2017 issue of the Journal of Democracy.
The authors, Yascha Mounk and Roberto Stefan Foa, believe they have found a set of leading indicators whose values may be able to predict whether a democratic form of government will go into decline in the near future. Here’s the money pull-quote:
Support for autocratic alternatives is rising, too. Drawing on data from the European and World Values Surveys [sic], the researchers found that the share of Americans who say that army rule would be a “good” or “very good” thing had risen to 1 in 6 in 2014, compared with 1 in 16 in 1995.
That trend is particularly strong among young people. For instance, in a previously published paper, the researchers calculated that 43 percent of older Americans believed it was illegitimate for the military to take over if the government were incompetent or failing to do its job, but only 19 percent of millennials agreed. The same generational divide showed up in Europe, where 53 percent of older people thought a military takeover would be illegitimate, while only 36 percent of millennials agreed.
There are many problems with this research. I’ll discuss three of them here:
- The research uses, in part, the World Values Survey and European Values Study. It happens that I have quite a bit of hands-on experience with this dataset. And it has issues.
- The Journal of Democracy does not adhere to standard academic publishing practices. It is not peer-reviewed.
- The authors have no apparent qualifications to conduct empirical research.
I’ll look at these in the order listed above. Hyperlinks should let you jump directly to a section.
The World Values Survey and European Values Study
Ronald Inglehart
University of Michigan Professor Ronald Inglehart is one of the founders of these surveys. He has posted a response to the authors’ earlier paper here. Or you can just click the screen capture below to see a readable image.
(click for larger image)
In the distant past I spent a few years on some research that eventually hit a dead-end. In large part this was caused by the nature of the data in these two data sets.
The first problem is an inconsistency. The wave definitions are different for the World Values Survey (WVS) and the European Values Study (EVS).
Wave Date Ranges (click for larger image)
The second problem is that not every country is surveyed in every wave. The Times article cites data for six countries. The table below shows the year in which each country was surveyed in each wave. New Zealand is an especially egregious case, with responses in only two of the six waves. (Click here for a pdf file containing the complete list of countries, waves, and years.)
Country Survey Dates (click for larger image)
[pullquote]In cohort analysis, one has to sort out life cycle effects from generational change and period effects. The latter take into account the fact that the current situation changes from one time point to the next.[/pullquote]
Strictly speaking, data used from different years should not be compared directly. Here’s what Prof. Inglehart pointed out in his email to me. ⇒
For example, consider the first wave (1981-1984). In the U.S. this was a period of sharp disinflation with the unemployment rate briefly exceeding ten percent. And, of course, 1995-2000 was the dot com boom. People were likely to be pretty optimistic during this period. Here’s what the Times story says about the periods compared:
Support for autocratic alternatives is rising, too. Drawing on data from the European and World Values Surveys [sic], the researchers found that the share of Americans who say that army rule would be a “good” or “very good” thing had risen to 1 in 6 in 2014, compared with 1 in 16 in 1995.
The global economy was doing pretty well in 1995. By contrast, 2014 was a year of economic stress. People are more likely to turn to authoritarian solutions during periods of economic malaise. That is a more plausible hypothesis for the authors’ results than a declining faith in democracy. And this is part of what Prof. Inglehart calls “period effects.” Simply looking at the numbers is not enough.
The Nature of Data
[Section removed per introduction at the top of this piece.]
The Journal of Democracy
Their full paper won’t be published until next month. It will appear in the January, 2017 issue of the Journal of Democracy. It is hardly ever a good sign when researchers publicize their findings in advance of publication.
And there’s even a problem with that. The Journal of Democracy is not peer-reviewed. Here’s their statement on submitting unsolicited manuscripts:
Although the majority of its articles are commissioned by the Editors, the Journal of Democracy also welcomes unsolicited submissions. Every unsolicited manuscript is read and evaluated by at least two members of the editorial staff. Many unsolicited manuscripts are rejected at this stage, often for reasons of focus, timing, style, or space that do not necessarily reflect on their scholarly merits. The Journal is not formally peer-reviewed, but in some cases manuscripts are sent to outside scholars or specialists for comments and evaluation.
Peer reviewing is the gold standard for academic publications. Publishing in a journal that does not use double-blind peer reviewing makes the entire body of research suspect.
The Authors Are Not Qualified
The authors, Yascha Mounk and Roberto Stefan Foa, have Ph.D.s in Government from Harvard. Dr. Mounk is class of 2015, Dr. Foa is 2016. Their pre-doctoral studies focus on philosophy and history. I could find no evidence that either has significant training or experience working with data. But Dr. Mounk sure gave a great pull-quote to the Times:
Yascha Mounk is used to being the most pessimistic person in the room. Mr. Mounk, a lecturer in government at Harvard, has spent the past few years challenging one of the bedrock assumptions of Western politics: that once a country becomes a liberal democracy, it will stay that way.
Curiously, Dr. Mounk is a lecturer in Government at Harvard. But he has his union card. (In academia, a Ph.D. is usually required for admittance to the faculty in most departments.) Why isn’t he on a tenure-track appointment somewhere?
Roberto Stefan Foa is an assistant professor in Political Science at the University of Melbourne in Australia. This information is from Dr. Inglehart; the University of Melbourne website is sorely in need of updating.
Conclusion
I look forward to reading the full article when it is published. Stay tuned.
