The problem with our reporting about polls

[iemanja]

The problem is less the polls themselves then in how the media report the polls.
I suggest DUers keep this is mind rather than working themselves up over what they see as RW bias in polls and in aggregating sites like 538. You don't need to be so eager to dismiss the polls because they don't say what the media tells you they do. They also don't say what you likely think they do.

You might wonder why, after the 2016 and 2020 presidential elections, journalists would put any stock in pre-election polls. But the biggest challenge for journalists is not in the polling itself, but how we report polling results.

If you look at the average of all of the political polls that Real Clear Politics tracks, you will find of the dozen hot U.S. Senate races only three of them are not a statistical tie right now. How can I say that when nearly all have a candidate leading by more than the margin of error? It is because journalists are not correctly considering what the margin of error means.

In a two-person race with a poll that has a three percent margin of error, a candidate would have to be leading, not by three points, but by more than six points to be a clear leader. Because remember, the margin of error could be three points in either direction for either candidate. Some polls are four percent margins of error. A candidate would have to be a blow-away leader to be more than eight points ahead.

And yet I see journalist after journalist reporting about candidates that moved up one point and that is big news. It just means the race is still within the margin of error, as it probably has been most of the year.
We put way too much focus on polling as if they will predict the outcome of races that are statistically too close to call.

https://www.poynter.org/reporting-editing/2022/the-problem-with-our-reporting-about-polls/

[Pantagruel]

1. Ads sell much better

for a horse race rather than a walkover. Very simple.

[iemanja]

7. It's certainly easier for them

They don't have to spend time thinking about issues and presenting them accurately.

[Farmer-Rick]

2. Exactly,

Half the time corporate media never even tells you the margin of error.

People don't realize that the margin doubles when comparing either sides of the question.

[Lovie777]

3. Seems like for corporate media and polling outfits have to follow the ingrained pattern ....

the opposite party "always" wins the mid-term election.

[Emrys]

4. This OP should be a sticky.

Many if us have been banging away about this for years about polls wherever they are. A point or two here or there is noise, that's it. A trend may be more significant, but that takes time to emerge. Key para.:

In a two-person race with a poll that has a three percent margin of error, a candidate would have to be leading, not by three points, but by more than six points to be a clear leader. Because remember, the margin of error could be three points in either direction for either candidate. Some polls are four percent margins of error. A candidate would have to be a blow-away leader to be more than eight points ahead.

[iemanja]

12. Thanks.

With so much concern about polls, it’s important to know how to read them.

[gristy]

5. Here's a good article that supports and explains your point

https://www.pewresearch.org/fact-tank/2016/09/08/understanding-the-margin-of-error-in-election-polls/

1. What is the margin of error anyway?

Because surveys only talk to a sample of the population, we know that the result probably won’t exactly match the “true” result that we would get if we interviewed everyone in the population. The margin of sampling error describes how close we can reasonably expect a survey result to fall relative to the true population value. A margin of error of plus or minus 3 percentage points at the 95% confidence level means that if we fielded the same survey 100 times, we would expect the result to be within 3 percentage points of the true population value 95 of those times.

The margin of error that pollsters customarily report describes the amount of variability we can expect around an individual candidate’s level of support. For example, in the accompanying graphic, a hypothetical Poll A shows the Republican candidate with 48% support. A plus or minus 3 percentage point margin of error would mean that 48% Republican support is within the range of what we would expect if the true level of support in the full population lies somewhere 3 points in either direction – i.e., between 45% and 51%.



2. How do I know if a candidate’s lead is ‘outside the margin of error’?

News reports about polling will often say that a candidate’s lead is “outside the margin of error” to indicate that a candidate’s lead is greater than what we would expect from sampling error, or that a race is “a statistical tie” if it’s too close to call. It is not enough for one candidate to be ahead by more than the margin of error that is reported for individual candidates (i.e., ahead by more than 3 points, in our example). To determine whether or not the race is too close to call, we need to calculate a new margin of error for the difference between the two candidates’ levels of support. The size of this margin is generally about twice that of the margin for an individual candidate. The larger margin of error is due to the fact that if the Republican share is too high by chance, it follows that the Democratic share is likely too low, and vice versa.

For Poll A, the 3-percentage-point margin of error for each candidate individually becomes approximately a 6-point margin of error for the difference between the two. This means that although we have observed a 5-point lead for the Republican, we could reasonably expect their true position relative to the Democrat to lie somewhere between –1 and +11 percentage points. The Republican would need to be ahead by 6 percentage points or more for us to be confident that the lead is not simply the result of sampling error.

In Poll B, which also has a 3-point margin of error for each individual candidate and a 6-point margin for the difference, the Republican lead of 8 percentage points is large enough that it is unlikely to be due to sampling error alone.

[iemanja]

6. Thanks. That's very helpful.

[gristy]

8. I think I got it wrong. I can't agree with the OP or with the article that I posted.

95% confidence in the polling result for each candidate being within the stated MOE leads to 99.75% (1-(1-.95)^2) confidence for 2x the stated MOE when applied to the difference in polling result for the two candidates. And the MOE that applies to a 95% confidence in the polling result for the difference in polling result will be the same as the MOE that applies to the polling result for each candidate.

[Fiendish Thingy]

10. You got it right- the article is correct. Nt

[iemanja]

11. But how can we assume 95% confidence?

For many polls? They aren’t all Pew. I assume Pew is explaining its own polls.

[Fiendish Thingy]

9. Excellent summary - a candidate's lead has to be at least *twice* the MOE

To be safely and decisively beyond “statistically tied” territory.