Assessing the facts after the Presidential Election: How did (some) pollsters get it so wrong? And what can businesses learn from their mistakes? 

If you are like me, you are glad to see that the election is over. No more negative ads and partisan bickering (at least for now). So if you are expecting me to do some politicking in this article, then you won’t find it. Instead, I want to understand something that should be fairly straight forward: how opinion polls are taken and the results interpreted. I also want to discover why so many polling “experts” got it wrong when they called for a close electoral vote. Finally, I want to understand if there are lessons from the election that I can apply to my business.

While it is true that politicians and media outlets employ professional pollsters and statisticians who are supposed to accurately and impartially predict outcomes of elections, it became inherently obvious that there were many versions of “accurate and impartial” during this last race. So where is the disconnect?

Let’s look at the first source for polls, the campaigns themselves. Campaigns use polling data to make decisions about two things: 1) how to craft and change their message to have the biggest impact and 2) where to deploy their resources. Over time, pollsters have learned what questions to ask, who should be asked, and other methods to reduce their margin of error, which in theory will accurately predict the outcome of elections.

The problem is that a third use of polls by campaigns is to change opinions and persuade voters. Campaigns regularly release polls that favor their side (and withhold those that don’t). As a result, a skewed picture is often disseminated to the public. In other words, you can’t trust the campaign’s pollsters to tell you the whole picture.

The second source for polls, the media, should be more reliable because theoretically, they are supposed to be impartial. But the real world tells us that there are numerous exceptions to impartiality. It is also true that the quality and reliability of certain polls can vary from media outlet to media outlet.

Then if the truth is out there, where should one look? I think the answer starts and ends with data.

In the not so distant past, pollsters had limited data to work with and substandard tools and methodologies. It was difficult and costly to get a clear and broad data set that could be statistically accurate beyond a reasonable doubt. So campaign strategists relied on a mix of data and seasoned campaign managers who interpreted the data and made “gut” calls.

In today’s world, data is much easier to collect and analyze. The explosion of big data and the development of tools and techniques with which to deal with this data has resulted in the ability to get polls right. We are in the middle of a transition period where the use of big data is becoming more science than art. Statisticians, experts in applied analytics, and mathematicians will replace the guys who used to make “gut” calls. As we transition, however, it will be difficult to know who’s who.

So here’s what I did. I researched who got it right on Tuesday. One name came up over and over again: Nate Silver. Silver is a statistician, journalist and psephologist (someone who uses scientific analysis to understand elections). While some have criticized Silver for being an Obama supporter and, as a result, questioned his objectivity, I believe he is a statistician first and foremost. This is borne out by the fact that Silver has an amazing track record predicting elections. In 2008 he correctly predicted all of the Senate races and only missed one state in the Presidential race. This presidential election his predictions were 100% accurate.

How does he do it? Through hardcore data analysis, proven statistical techniques, and some proprietary methodologies that he discovered while creating a forecasting system centered on baseball stats, Silver has developed a reliable predictive model. Silver’s model is not perfect (none ever are), but it removes much of the bias and imperfections based on gut calls that are typical of most of the prognosticators out there. . You can find Silver’s website at

From a business management standpoint, one can learn a very valuable lesson from this. Virtually every day we read about “Big Data” and how it is enhancing intelligence, business related or otherwise. Businesses are creating vast amounts of data and are studying, interpreting, drawing correlations and making strategic decisions based on this data. Businesses which run on gut, do not practice sound statistical analytics, or do not leverage the insight that data gives them are doomed to failure in this age.

So if you wondering how to make better business decisions and improve your business’s results, hire someone who is an expert in applied analytics and unleash the power of your database. Those who deny the facts of data that sits right in front of them will ultimately fall behind.


  1. Michael,
    Thanks for you blog. Excellent points. I followed Silver along with the Iowa Capital Market throughout the election campaign. Although using different approaches both consistently outperform the pundits and any individual poll.
    To criticize Silver because of his political leanings show that the critics just don’t get it. They can criticize his models, his calculations or his data if they can find legitimate reasons. But to simple claim he is biased because of his politics without offering any evaluation of his methods is like rejecting the Theory of Relativity because Einstein was an atheist. (Which many people did at the time. I would hope we’ve moved beyond that – but maybe not.)

    I think there are 2 reasons many pundits were critical.
    1.They know their political standing affect their prognoses, so assume the same of him.
    2.In his book, the signal and the noise, he dares to track the pundits forecast; they do about as well as monkeys with a dart board. Probably didn’t win friends in the pundit class.

    I teach a graduate class in forecasting. It’s been established over the last 20 years or so that if you take a group of independent forecasts (it works even if the forecasts are not independent, but not as well) the average of the forecasts will be better than any of the forecasts. Not better than some and worse than others the way we usually think of averages, but better than all. I’ve written an article on why this so (not yet published). I know Silvers’ method is more advanced than just averaging polls, but, as I understand it, that’s the heart of it. He’s on sound theoretical ground as well demonstrated results.

    Another way in which big data and analytics was a winner Tuesday. The Obama campaign made extensive use of data bases and analytics to mirco-target votes. It apparently worked, getting record turnouts where he needed them, despite predictions that the enthusiasm gap favored the opposition. Roosevelt was called the first radio president, Kennedy the first TV president. Will Obama be the first analytics president? Doesn’t quite have a ring to it, but perceptions change. Maybe the first Moneyball President would have more popular appeal.

    Stephen W. Custer Ph.D.
    Assistant Professor of Management
    Teaching Forecasting and Analytics
    Virginia Commonwealth University
    Richmond, Virginia

    • Very interesting points Dr. Custer. Your labeling of Obama being the “first analytics President” is great. It certainly marks a major shift in the way campaigns will be run going forward. Thank you for your comments.

    • Hi Steven

      You write : “if you take a group of independent forecasts (it works even if the forecasts are not independent, but not as well) the average of the forecasts will be better than any of the forecasts.”

      Would you happen to know what happens if one takes the Median ?

      Thank you !

  2. Michel,
    This is Steven’s reply to your question:
    I’m not sure if you mean taking the median of several sample or surveys and then averaging them; or taking the median of three or more forecasts. For both cases I have not done or am aware of any studies. My guess (note it’s an informed guess, but still a guess) in the first case you would get similar improvements as for averages. Part but not all of the reason averaging forecasts generally improves the forecast is that it is equivalent to increasing the sample size. This would hold for the median as well as the average.

    In the second case where you select the median of several forecasts; it’s not a bad idea but it can’t give the improvement of averaging. Frequently the forecast obtained by averaging is better than all the forecasts that make up the average – count intuitive isn’t it? Selecting the median forecasts can’t be better than all the forecasts – clearly not the median forecast. I’d stick with taking the average failing any definitive studies.

  3. Pingback: THE TEN MOST POPULAR POSTS FROM 2012 | The More You Know BBlog·


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s