Stunning, yes, but there were clues: Why polls were not wrong

About every New Hampshire poll released in the run-up to yesterday's primary showed Barack Obama leading by significant margins. It was thus to be expected that, after Hillary Clinton's stunning victory, poll-bashing would become everyone's favorite pastime. Most observers are lamenting how they were misled by polls, and using them as cover of their deficient coverage.

Yet, the polls did not get this election wrong. A careful reading of the numbers that were being released should have made anyone weary of predictions, as the New Hampshire electorate looked to be in flux. Obama was rightfully considered the favorite, but the results should have been considered unpredictable. But many in the media -- MSNBC is a prominent example -- are now trying to blame pollsters, when what should be faulted is the way they chose to interpret these surveys and use them to run with the storyline of an unstoppable Obama.

First, a few polls released late Monday and throughout the day Tuesday showed Obama's bounce stabilizing and Clinton starting to reverse the trend; this was especially the case in the last Rasmussen and ARG polls. Add to this the fact that most polls released on Monday that showed double-digits Obama leads were in the field over the week-end. That means that not only was at least half of the sample tested prior to the ABC debate, but most of the interviews were conducted at the height of the post-Iowa bounce, when voters were still processing the Iowa numbers.

The time allowed for campaigning in New Hampshire was five days; could it really have been expected that surveys taken on Saturday (the second day) would give an adequate picture of what would happen on the fifth? The very fact that time was so compressed guaranteed that the electorate would shift from day-to-day and in much higher proportions than in a typical election, and the swings were perfectly predictable given the race's timing.

In addition, bounces are intrinsically volatile, one of the famous examples being McCain's sudden and short-lived South Carolina surge in 2000 in the days after his New Hampshire victory. Here again, Obama jumped 10 to 15% in the space of 48 hours and no bounce like that can be sustained over time.

The question was never whether Obama could make that bounce permanent but whether the surge would last long enough to carry him through the finish line. I thought, like most observers, that five days was way too short a time for Obama's lead to start eroding and did not believe that Clinton could stage a come back. At the end of the day, she managed to crush Obama's momentum by Tuesday. But what is surprising is not the fact that she did but the rapidity with which she did it.

Clinton needed every single one of those five days to press her case and, with the clock ticking, she was helped by the intense media coverage and the accumulation of very publicized events, in particular Saturday's debate and "The Tears." There is no way that any poll in the field over the week-end, at the height of the Obama buzz, could have picked-up Clinton's come back.

Finally, the big question mark going into the election was how independents would break down between the two parties' primaries. Would a larger proportion vote for John McCain or for Barack Obama? This question was deemed decisive in both contests, and indeed the fact of the matter is that many more than expected voted in the GOP primary. And that both propelled John McCain into a solid lead and saved Clinton's candidacy. It was obvious that both McCain and Obama could not have a big night; they could both win, sure, but two large victories as some polls predicted was out of the question since a key constituency for the two candidates overlapped.

Independents appear to have moved away from Obama in the final day as fast as they had jumped on his bandwagon. Polls had not picked this up but they could not have been expected to. For one, all surveys constantly pointed out that we should watch for independent breakdowns, explicitly stating that these numbers were the most volatile and hinting at the uncertainty they were reading among undecideds. Second, exit polls confirm that this move to the GOP race occurred in the final day, which is perfectly compatible with the volatility of Obama's momentum.

The most troubling aspect of the night is that coverage of polls might have had a direct influence on the results. Reading everywhere that Obama's victory was ensured, a large proportion of independents might have then decided to jump to the GOP race where McCain seemed in more trouble and where some polls showed Mitt Romney leading.

The media had all this information in its hand, yet it chose to go ahead and portray the Democratic race as a done deal. While I thought Obama would win yesterday like just about everyone else, I did explain in my pre-election guidelines that the polls had Clinton stabilizing and that a tight defeat is not at all out of the question. And, I added, "a Clinton victory tonight is still possible, and it would (justifiably) be treated as a huge come back."That the very same observers who trumpeted an Obama lead now are complaining about the treachery of polls is disingenuous.

The second storyline that is being used to deflect the blame is even more stunning, as some are invoking the Bradley effect to explain the polls' mistake. The Bradley effect contends that many white voters who would be uncomfortable voting for a black candidate might lie to the pollsters to not have to admit their racial instinct, and thus lead the surveys to overstate the support that candidate is getting. In other words, New Hampshire Democrats could not get themselves to vote for an African-American candidate once they were in the privacy of their voting booth.

Even if we admit that New Hampshire Democrats are so dramatically more prone to this racist instinct than Iowa Democrats, the Bradley effect explanation is discredited by every indication we have of what happened yesterday and the breakdown of voting groups. For one, pollsters were modeling more independents to choose the Democratic race that actually did; and given Obama's lead among unaffiliated voters and Clinton's among registered Democrat, that discrepancy goes a long way towards explaining the size of yesterday's swing.

Second, Hillary Clinton's come back was fueled by a shift of the female vote. The polls showed that Obama had overtaken Clinton among women after his Iowa victory. But exit polls showed that Obama ended up crushing Clinton among men but overwhelmingly losing females. In other words, the polls completely missed the female vote but not the male one, and that is very difficult to reconcile with the Bradley effect; white men are usually described as the group that feels the most threatened by racial dynamics, and if racist instincts kicked in here, it makes little sense that it would have only affected women. In addition, there are many very good explanations that can be offered for the swing of the female vote -- the tag-teaming against Clinton at the ABC debate, Clinton's display of vulnerability on Monday -- all of which fit the facts much better than recourse to the Bradley effect.

So do not be afraid of trusting polls in the coming weeks; just learn how to read them better.



  • I think it was absolutely absurd to hear CNN talking about the "Tom Bradley Effect" last night, as if to imply that New Hampshire was somehow inherently racist. What made me more annoyed was much time was spent over the past few days talking about how "Iowa is color-blind, maybe America is too." So, apparently, you can't be color-blind and be for Hillary... at that point, you're racist (as so implied). Ridiculous.

    Second, I sent that link to the Nashua Telegraph stories yesterday. When reading those throughout the day, you could tell that it was a more even split, anecdotally, for Independents. The stories of towns running out of ballots had mentions that they were of both ballots, GOP and Democratic and not lopsided for one over the other.

    Once again I appreciate and enjoy your analysis!

    By Anonymous app state, At 09 January, 2008 10:10  

  • Hi, I work for an international discussion programme called World Have Your Say on BBC World Service radio and today (Wednesday 9th) between 1pm and 2pm Eastern Time in the States we are discussing why the media and pollsters got it so wrong when it came to predicting the results in New Hampshire and whether the media is in love with the idea of an Obama presidential victory. If you are interested in taking part in our discussion please send meyour contact numbers to martin.vennard@bbc.co.uk or call me +442075570635 and I will call you straight back.

    Many thanks

    Martin Vennard

    By Blogger Martin Vennard, At 09 January, 2008 10:21  

  • Re: last-minute shift of women to HC.

    As an expat living in the EU and only now beginning to pay close attention to the campaign, I was hugely impressed with Gloria Steinem's op-ed in the NYT yesterday (listed as most popular op-ed piece): "Women are never the front runner". Got no idea whether NH voters are big time NYT readers, but GS did put out a compelling gender-based argument.

    Any way of testing any influence this piece might have had?

    GS is open in the article about working for HC, btw.

    By Anonymous Hamburger, At 09 January, 2008 10:31  

  • The Tom Bradley effect is actually a very possible influence here. Obama supporters keep telling themselves that race isn't a problem. But the results aren't supporting that notion.

    By Anonymous Anonymous, At 09 January, 2008 10:52  

  • I think Steinem's article was only what is to be expected from her. She tried to downplay the struggle of African-Americans to vote in an effort to garner sympathy for Hillary. Many blacks in America got their vote in the 1960s, not the 1860s.

    I'm glad she disclosed that she was working for HRC, but it's still obvious that gender had too much influence on her decision. This is beyond the fact that a woman like HRC who gained much of her fame from being married to a man who has publicly cheated on her is hardly an ideal example for a feminist.

    In the end, we should be voting for someone because of their personal positions and qualities, not their gender or race.

    By Anonymous Kathy, At 09 January, 2008 10:58  

  • If race didn't matter in Iowa, why would it matter in New Hampshire? Could it not be that voters simply thought Hillary was more qualified to be president? I believe exit polls actually said just that.

    By Anonymous Anonymous, At 09 January, 2008 11:06  

  • You said: "Obama's post-Iowa momentum was crushed in a matter of days."

    But given that before Iowa Obama was down 14 or more points, I'd say the Bounce stuck...or at least most of it.

    By Blogger David Dobson, At 09 January, 2008 11:07  

  • I wonder if the results vindicate Mark Penn's methods.

    By Anonymous wglad, At 09 January, 2008 11:11  

  • David, Obama was not "down 14 or more points" in the New Hampshire polls coming out just before Iowa. He had already started to move and was even leading Clinton in some polls. The bounce didn't stick much at all.

    By Anonymous Mr. Rational, At 09 January, 2008 12:27  

  • I also don't by the Bradley theory here. Obama still did very well with white men, so only women in New Hampshire are racist? C'mon.

    To add on to your idea that there were indeed signs, Obama actually finished around his pre-results polling average. It was just that Clinton did much better than expected at the expense of John Edwards.

    Excellent post, CD.

    By Anonymous Andy, At 09 January, 2008 13:44  

Post a Comment

Subscribe to Post Comments [Atom]

<< Home