Meta-Analysis, the Election, and 538

image_pdfimage_print

NPR’s “On the Media” show today featured Brooke Gladstone interviewing Nate Silver, lately infamous as the analyst and writer at the New York Times 538 blog.  Ms. Gladstone asked a question about meta-analysis, the election, and 538.  Paraphrasing, her question was, “Since you use meta-analysis and include a number of different polls, shouldn’t your analysis be more accurate?”

This is a common, incorrect belief.  It’s often called “regression to the mean.”  Statisticians will appeal to the central limit theorem.  Unfortunately, neither of those apply when it comes to election polling.  As a wise teacher once told me, “Errors don’t cancel, they accumulate.”  He was right then and is even more correct today.  Meta-analysis allows the errors in the various polling methodologies to accumulate.  It’s made even worse this year by the non-response rate to polls, hovering near 90 percent.  That means for every ten calls a pollster makes, they get one response.  So much for the “random sample” assumption.

I’ve made my predictions elsewhere.  I look forward to watching Mr. Silver eat a large portion of crow Wednesday.  For those interested, David Burge has written a clear, cogent description of what’s wrong with political polling.  Just skip past the math — the really good stuff starts about halfway down the article.