On polls and polling

[Stuckinthebush]

As the election gets closer, we are seeing more frequent polls and more frequent posts about the veracity of these polls. A number of DUers caution against the use of polls, and a number say that the polls can be useful when used correctly. Both camps are correct. The problem is with the media's interpretation of polls. When they cite a poll, they will say, "Bush has a strong lead over Kerry." This is the entirely wrong way to use polls. All pollsters will tell you that their methodology produces a substantial amount of error. The MOE is not always accurate because it is based on probability theory which assumes random sampling and that each member of the population has an equal chance to be in the sample. Neither of those assumptions are met with polling. Nevertheless, polls are amazingly accurate. Think about it...the science allows a pollster to call 1000 people and get a rather good picture of the overall sentiment of the population. Amazing, actually. So instead of bashing polls, we should admire the science and bash the poor use of polling data.

If we take a look at the final polls for the 2000 race, and then compare those results to actual(Gore 49%, Bush 48%, Nader 3%), we get the following:

Poll.........Gore Bush Nader...GoreDif BushDif NaderDif

ABC...........45...48....3.......-4........0.......0
CNN/TIME......43...49....3.......-6........1.......0
Gallup........43...47....5.......-6.......-1.......2
NBC/WSJ.......44...47....4.......-5.......-1.......1
Newsweek......43...45....5.......-6.......-3.......2
Pew...........43...46....3.......-6.......-2.......0
CBS...........44...46....4.......-5.......-2.......1
CBS/NYT.......42...47....5.......-7.......-1.......2
Fox...........43...43....3.......-6.......-5.......0
Harris........47...47....5.......-2.......-1.......2

Median Difference................-6.......-1.......1

What does this tell us? It shows that polling methodology is not adequately picking up the Democratic vote and understating it by 6 points. The Republican underrepresentation is only around 1 point and the Nader overrepresentation is around 1 point. This makes sense if you think about it. Pollsters can't get to the Democratic base as easily as the Republican base. Even if they weight their samples based on predicted voter turnout, the data are off by 6 points for Dems and 1 point for Republicans. Methodology is probably even more inaccurate these days because of an increase in exclusive cell phone users.

So, when we see a poll that shows a dead heat, rejoice, don't fret. When we see a poll that shows Kerry down by 4, rejoice, don't fret. Remember, polling is based on probability theory which assumes random sampling and the possibility that all members of the population have the same chance of being in the poll. This can't happen and the error is more evident in the Democratic side of the poll.

[Stuckinthebush]

1. A sad, self-serving bump

:shrug:

[swag]

2. Nicely done

a good read

[Stuckinthebush]

7. Thanks! (n/t)

[jefferson_dem]

3. Cogent analysis. I agree that a tie in the polls at this stage is good.

I, for one, totally respect the predictive validity of survey research. Of course, some pollsters weight their sample to more accurately reflect the partisanship of the electorate - the "gallup vs. zogby" approach has been discussed in multiple threads. The partisan imbalance seems to present a growing challenge to survey researchers.

I'm curious as to why Repukes are easier to get ahold of than Dems nowadays. Could it be that Dems are more likely to abandon their landlines and go cell phone only? That, could be our secret weapon on election day - the turnout of voters who are outside of the polled population of voters, who will vote disproportionately for the Dems.

[Stuckinthebush]

6. That is the million dollar question

Why are Dems harder to poll than Republicans?

Cell phones could be a factor. Demographically, Republicans make more money than Dems and have more one wage earner households. So perhaps more Dems are holding down two jobs, or jobs at odd hours of the day so are less likely to be at home. Perhaps Dems are less likely to answer a call that is from someone they don't know. Perhaps the economic backend of the Democratic curve is so much lower than the economic backend of the Republican curve that 5 to 10% of Dems can't afford the luxury of a land line and a cell phone, so opt for a cell phone only.

Good questions.

[secular_warrior]

4. My problem with the polls showing Dems behind

is that it can have a serious impact on the race itself, i.e. polls show the Repub candidate winning, people like to support a winner, actual support for Repub candidate goes up...

I understand your point about the scientific virtues polling - I think it's a great thing. But I also think the polling industry has to take a serious look at why they're constantly underestimating Dem support and what they can do to get it right. Also, there has to be much more responsibility as far as the media goes when it comes to reporting the polls - knowing how wrong they can be. When reporters such as Judy Woodruff come on and say "President Bush has pulled ahead by 8 points in our latest poll..", something like that can have a profound effect on public sentiment, and the race itself.

[Stuckinthebush]

5. I agree 100%

Especially with your contention that the media needs to exercise restraint in how they play polls.

But remember, organizations such as CNN, ABC, etc. pay large sums of money to have their own polls completed. They have a lot invested in the accuracy of their polls, and in their polls being seen as the most accurate. So in essense, they have a conflict of interest with their polling. They need to be journalist and challenge the veracity of everything they report, but they can't do that with their internal polls because they would be criticizing themselves. They put themselves in a very bad position.

As for pollsters needing to better understand why they have such poor Democratic representation...I think they do try to do that. They are victims of our society just like you and I are. They rely on telephone calls (or the internet in Zogby's case) to get their sample. They can not call cell phones, so they are stuck with land lines. I know for a fact that they try to oversample certain neighborhoods and areas that may give them a better Democratic sample, but if people don't answer the phone or are not at home or don't have a land line in those oversampled areas, then the pollster is back at square one.

You'll notice that both campaigns have their own internal pollster and these data are never or rarely given to the public. I assume that the internal pollsters use larger samples and more stringent screening methodologies to get as best of a picture as they can. This costs a lot of money, so you don't see a Gallup doing the same due to the large number of polls they spew out on a weekly basis.

[secular_warrior]

8. Maybe part of an electoral and media reform agenda

should be to not allow polling companies to be owned by media companies, so there will be no conflict of interest. The media companies could then report and challenge ALL major polls, which will always give a more accurate picture.

I understand that it's probably more difficult to poll Dems, many of whom tend to be poorer and off of society's radar. But then polls should always be reported with the disclaimer that in all likelihood Dem support is being underestimated.