I'm a big fan of Nate Silver -- he consistently demonstrates that he is one of the best and brightest statisticians around. I like to say that statisticians (and risk managers) are professional skeptics; our job is to let data speak for itself, not to speak on its behalf. Nate Silver does that better than anyone. Through a combination of clear visuals, transparent methods and clear writing, he systematically works to remove biases of any form from his chosen medium of poll numbers.
Well, it turns out there are politics in everything -- including analyzing politics. A number of articles were written this week accusing Silver of not only failing to do his job but also obfuscating reality for his own agenda. Dean Chambers kicked it off with an oddly personal rant and others like Dylan Byers jumped on the bandwagon shortly thereafter. All of the authors have in common a poor grasp of statistics and an even worse understanding of its objective: to present otherwise-compromised or noisy data in as clean an environment as possible.
The articles were met, at first, with vitriol from the Silver defenders (it can't be ignored that the two sides in this drama fall along party lines). Fortunately, cooler heads have prevailed and a number of wonderfully clear pieces have been written defending Silver's methods and sometimes even his conclusions. In particular, I liked Scott Galupo's take, Ezra Klein's opinion and this post on Simply Statistics. If there's a silver lining (pun most certainly intended) in this outpouring of "Shut up, nerd!" comments (to borrow from Mr. Galupo), it's that it provides a teaching opportunity for something that really does seem hard to understand at first ("How can he lead by 0.5% and have a 75% chance of winning!?") but turns out to be quite simple and even familiar -- I think Silver's football score analogy is fantastic.
So that's the background. Things got quite a bit more interesting today when Silver wrote the following on the NYT's 538 blog, in a post simply -- and provocatively? -- titled, "For Romney to Win, State Polls Must Be Statistically Biased":
Nevertheless, [wishful thinking is] potentially more intellectually coherent than the ones that propose that the leader in the race is “too close to call.” It isn’t. If the state polls are right, then Mr. Obama will win the Electoral College. If you can’t acknowledge that after a day when Mr. Obama leads 19 out of 20 swing-state polls, then you should abandon the pretense that your goal is to inform rather than entertain the public.
So it would appear that we've passed the point of defending statisticians, burst through the level of explaining statistics, and are now fully on the offensive! A lot of commentators are calling this a very aggressive shot. I still think Silver is playing defense -- but he's doing it with a crowd that will never see it that way. Silver's objective is to map early polling data to actual electoral outcomes. He has a model that performs that transformation. And for some time, his model has been saying that if the polling data is good, the outcome is far from a toss-up. That connection is key: if the polling data is good, the model's conclusions likely hold. Silver isn't claiming some crazy left-field insight based on proprietary information; he's simply saying that after considering all available polls, it appears much more likely that the President will be reelected than not. It's the same set of inputs and same set of possible outputs that every single pundit in the country is considering. The only difference is that Silver has tried to quantify the degree of uncertainty, and he's found that it's different than you might suppose.
This isn't magic; this is just a statistician explaining what the data says, assuming the relationships that data represented in the past will continue to hold at this time. I admire Silver for being brave enough to put his money where his mouth is even though he most certainly knows that all the probabilistic hedging in the world won't save his reputation if the incumbent doesn't win. The odds may be 3-to-1 against, but as soon as the probabilities collapse to a certainty, that will be forgotten. He'll either be a visionary or a false prophet. In a way that's sad, either way, because in so many cases the distribution of possible outcomes can be more interesting or valuable that the one that actually materializes, and I wish we could get people to focus more on that idea. But realistically, you can't ignore the outcome.
We talk about "garbage in, garbage out" all the time, and this is a fantastic example. Silver has pulled apart all the relationships that used to hold for electoral data, but if there is something wrong with his input data, all the modeling in the world won't save him. It could be something as simple as cell-phone users (and therefore younger people) being underrepresented in modern polls; it could be something as complex as outright fraud. It could be nothing. Ultimately, Silver's job is to let the data speak. On Tuesday, we'll find out how much of what it says is true.