ALL POLLS ARE INHERENTLY FLAWED....

[Lerkfish]

(this seemed worthy of its own thread)

There are three areas where the best conducted polls are inherently flawed: Sample, methodology, and conclusions.

SAMPLE: When looking at a poll, pay close attention to the SIZE of the sample. For example, a poll of 500 people is not large enough to predict the voting of 200 million people...its statistically insignificant. The smaller the size, the more likely is the sample population unrepresentative. Although even large samples can be artificially skewed (see methodology below).
Also pay attention to the MAKEUP of the sample. Good polls provide a demographic breakdown so you have some idea whether its representative. However, even a sample that appears to represent the demographics of the larger whole, can still be artificially skewed by such factors as is discussed in methodology below.

METHODOLOGY: there are many things in this category, all of them important factors in how polls can be skewed:
-- WILLINGNESS TO BE POLLED. Consider: if you are a republican, and the Democratic Party calls you for your opinion, would you stay on the line? If Fox news calls, someone who watches fox would be more likely to sit still for a poll than one who does not. That automatically skews the sample to those who are first of all predisposed to participate in polls in general, and secondly inclined to participate in a particular poll.
-- ACCESS TO POLLSTERS. Many polls are conducted via telephone. That automatically culls the sample by work shift, access to a phone, whether your listed or not, etc. Also, the methodology of where those phone numbers were obtained is important. For example, is it a subscriber list, a random phone dialing, etc. If the poll were about how people felt about being out of work, and its a telephone interview, how accurate could that sample be if most unemployed people have no phone or even a home?
-- QUESTIONING. This one of the first things I'll mention that has the worst abuse. How a question is asked, and what questions get to be asked is subjective on the part of the pollster and can really reflect a bias in the organization funding the poll. A relatively benign example was a poll that was conducted last year that pointed out that teenagers could not locate certain countries on a map. It made the kids look really stupid until you noticed who sponsered the poll : National Geographic.
Here's an example of how the apparent same question can elicit different responses, depending on how it is asked:
------1. Do you think Bush is doing a good job? yes or no.
------2. Do you think Bush is doing a horrible job? yes or no.
------3. Do you think Bush is performing well as commander in chief? yes or no.
------4. Do you thing Bush is a good president?
------5. Would you prefer to have Bush as president?
------6. Is Bush the best choice for president?
you see, even though this seems to be asking the same question, the same person might not answer the same for #2 as they would for #6.
(there is actually more to this part, but you get the idea)

CONCLUSIONS: Even the most honorably conducted poll can have someone analyze and spin the data in such a way to fit an agenda. Here is an example:
a recent poll of journalists found that, in their opinion, they consider themselves:
---65% Centrist or moderate
---25% Liberal
---10% Conservative
Now, someone who thinks there is a liberal conspiracy will state that there are more than twice as many liberals than Conservatives (which is accurate, as far as it goes, though it neglects to say "of those expressing a side"), OR you could go so far as to say that only 10% of newsrooms are Conservative, and that proves 90% of them are liberal biased.
(this, btw, is a real life example of an argument made by a republican on another board. No matter how much I pointed out that well over half the respondents claimed no affiliation either way, he kept insisting 90% were liberal)
Another real life example is that there was a study to show the "fitness" of teachers to be teaching. Most of the parameters made sense, like their level of education, etc. But one parameter was "does the teacher belong to a union or teacher's organization?". If they didn't , that was one parameter AGAINST them. The study was sponsored by a teacher's union. Their data was accurate, but they were biased in how they analyzed or collated the data, because the union question held as much weight in their 5 point criteria and whether the teacher had an advanced teaching degree.

at any rate, at BEST polls are a rough indicator, but they are never conclusively representative of the whole, only of the limited sample. They should always be taken with a healthy dose of salt, even when they tell you what you want to know...probably ESPECIALLY when they tell you what you want to know. :)

[Beetwasher]

1. Yes, Of Course, Which Is Precisely Why They Are Easy Prey

for unscrupulous organizations to use as propoganda devices.

[olddem43]

2. It's the Heisenberg Uncertainty Principle, as applied to politics

You can never measure anything perfectly, because the very act of measuring affects the result.

[Courtesy Flush]

3. And do they screen likely voters?

They don't always use the phrase "likely voters", and I'm not sure how they determine what makes a person likely to vote. To me, they can say they're gonna vote, but I want to know if they have voted in recent elections.

During an election year, a popularity poll doesn't mean jack. It's the real voters whose opinions count. In my own experince, some of the most politically opinionated people I know, never vote.

[newyawker99]

16. Hi kdsusa!!

Welcome to DU!! :toast:

[porphyrian]

17. I think they use people who are registered and...

...who have voted before, though this depends on the specific poll in question.

I would just like to add that the people who perform telephone surveys are often private contractors. The interviewer are usually paid shit and are treated poorly. Their jobs often depend on producing results. Do you really think data collected in this manner is accurate? It doesn't take a genius to know when you can fudge a survey or two without getting caught, and the people who do usually aren't geniuses. Check the turnover rate of your local phone sluts if you doubt what I'm saying.

[Name removed]

4. Deleted message

Message removed by moderator. Click here to review the message board rules.

[linazelle]

8. You tell 'em prolesunited....all of sudden polls are irrelevant

They weren't irrelevant or useless when Bush was leading :eyes:

Do they EVER stop trying to deceive? :shrug:

[Lerkfish]

11. um...dude?

I'm a democrat. I just work with polls all the time as a newspaper graphic journalist. I just wanted to share the pitfalls of polls, in general. I have to look at this type of data, and over the years, this is what I've found wrong with polls in general.

did you even read what I had to say? When I talked about how republicans I'd run into on other political forums didn't listen when I explained their skew on the data?

[Lerkfish]

12. nope

I WANT kerry to win. I attended the Dayton Kerry rally, and have posted about it.

what is wrong with you people?

[TruthIsAll]

5. Not one word about the poll MoE.

Sure, polls can be rigged. My current thread on the subject concerning the CNN, AP and FOX Kerry/Bush polls show a clear bias toward Bush when compared to 10 other polls for which I also have monthly data.

FOX and AP do not show Kerry leading Bush in ANY of their monthly polls so far this year.

CNN/Gallup had Kerry up by 8 points in March. And down 5 points in April. How can you trust this poll, especially in light of the wild gyrations it had week to week in 2000?

Other polls, though, have good track records. They include Zogby, ARG, CBS and others. The trick is to know the good ones. And to be leery of the bad ones.

It's no secret that CNN and FOX have been pulling the wool over the eyes of millions of Americans when it comes to concealing the crimes of BushCo. To rig the polls in his favor seems like the logical thing for them to do. Its just whoring by other means.

It is in their interest to keep Bush close to Kerry. That's not conjecture at this point. It is a fact. To deny it at this point in time, after all the evidence of Bush media-whoring by these cable-whores over the last four years, is just plain nuts.

[Enraged_Ape]

6. Not to mention that the reporting of polls adds a whole new layer of BS

For every poll that is reported, there are probably at least two or three others (and who really knows how many?) conducted that don't give the results that either the client of the poll or the reporting organ is looking for. If you draw enough random samples of people, you will eventually come up with a majority of dunderheads who will state that Bing Crosby's ghost visits them every night. Report that poll and you will make it look like the majority of Americans really do think that Der Bingle visits them in their beds, even though it's horseshit.

Bush is getting all the help he can possibly get in the reporting of these polls. There is no way that almost half of America thinks he's doing a great job. I hang around with a crowd that runs from moderate Democrat to conservative Republican, and I don't know one single person that will vote to elect the numbskull in November.

[xocolatl]

7. Polls serve a useful function

They let us know what to expect election day. Would you rather be totally ignorant up until the day of the election?

Of course a poll is only as accurate as its underlying methodology, and certainly there are organizations that would be unscrupulous in their polling practices in an effort to manipulate the outcome. That doesn't invalidate the exercise. It just means that you, as an individual, have to take responsibility for interpreting the results, which is what you should do with any published material.

I'm not knocking TruthIsAll in the other poll thread. I think he has done a valuable service in collecting the data, giving us the opportunity to quantify differences between different organizations.

But I react to strong statements such as "I've proven there there is a bias", especially when the stats are less than convincing. It turns out that he's right, if you do the stats more carefully (although I disagree with his assessment on CNN).

[West Coast Democrat]

9. Didn't you already post this in a different thread?

[Lerkfish]

10. yes, sorry...

I thought it could be its own topic, rather than tied to a specific poll, since it discusses polls in general.

If that was wrong, I apologize.

[Eric J in MN]

13. some polls adjust by politcal party affiliation

If too many respondents say "I'm a Democrat and I'm voting for Kerry," then they will "weight" the poll on the assumption that there aren't a lot of Democrats and declare that it's a close race.

[JackRiddler]

14. The most significant poll finding of all time

About 70 percent of all persons polled in all polls do not complete the poll - they hang up, or simply go away if on the street - and thus are not included in the results.

Pollsters and academics like to produce studies (arguments, really) showing that the poll results would be the same if all those sampled participated. These are little more than weak attempts to legitimate their professions.

There is no way to say how these 70 percent feel. The group who refuse to participate will vary from poll to poll. People liable to hang up will tend to have a different personality and hence outlook than those who stay on the line. There is no reliable way to weight the 30 percent who do respond so that the results correspond to what the 100 percent really think.

Here is an excellent article from retropoll.org, a classic in debunking the bad science typical of opinion surveys.

http://retropoll.org/polling_fraud.htm

Polls usually report out a statistical "margin of error" for their results. The margin of error that polls report depends not upon the number of people called but upon the number who responded, the sample size. They usually report a margin of error of about 3% for a sample size of 1000. But this margin of error statistic that makes polls look highly accurate is, in essence, a cover to hide the 70% who refused to participate. Even if 99% refused to participate and we had to speak to 100,000 people to find 1,000 who would talk with us, the margin of error statistic would still be reported as the same 3%. It would be hiding the problem of non-responders. So the margin of error statistic is not only inappropriate in this circumstance; it suggests a level of certainty that is fraudulent.

While it is always possible that those refusing have similar views to those agreeing to be polled, Retro Poll has found evidence to the contrary. When we asked over a thousand people, "would you take a few minutes to respond to a poll on the impact of the war on terrorism on the rights of the American people", one woman responded: "You wouldn't want to hear our view on that. People wouldn't like what we think."

"That's ok", we said, "your views are important; they should be counted and reported as part of the democratic process. We want your opinion to count." "No," the woman said insistently. "We're against the war the way they did it. We think they should just bomb all of them, not send our troops over there...." We didn't ask whether she meant bomb everyone in Iraq or some larger group of Muslims, or nations of people, but the woman's self-awareness that her views were outside the "norm" caused her to refuse to participate. Undoubtedly others have specific and different reasons for non-participation that we have difficulty ascertaining because most won't talk about it.

If the "bomb them all" couple may seem the exception among non-responders, consider this: Fewer African Americans and Latin Americans agreed to be polled in both of our national samples (in the current poll 5.7% were African Americans and in the prior poll 4%; for Latin Americans the corresponding figures were 6.2% and 8%. Each of these groups make up about 12% of the U.S. population, actually 12.5% for Latinos). As a result, our poll sample ended up at 79.4% "caucasian" ( i.e. European American) but the actual White/non-Hispanic European American proportion of the population is 69.1% according to the 2000 Census.

http://retropoll.org/polling_fraud.htm

[porphyrian]

18. Absolutely! Sample Size and Response Rate! - n/t

[TOhioLiberal]

15. Especially the internet ones...

How many threads with 'DU this poll' and 'Defreep this poll' have you seen?

[JackRiddler]

19. indeed

Clicking on these bogus Internet polls has got to be the most useless pastime of all. We should be thrilled if the Freepers are wasting their time freeping polls. Forget it, it's nothing more than a penis length contest. If you want to do something constructive, turn your favorite posts into leaflets and go distribute them at a mall!

[patcox2]

20. Only one disagreement

Sample size. You are right that 500 is not enough to get a reasonable margin of error. But the way you are saying it seems to perpetuate the myth that, say, 1,000, is too few people for an accurate sample. Thats just not true. Anywhere above 800 is enough, I know its hard to beleive, but its a function of the mathematics. 1,000 really is enough to make your statistical error smaller than your other errors (at least in measurements near the middle of the scale, the margin of error for data that occurs less frequently (like Nader votes) is higher than the margin of error for Bush and Kerry votes, which are more evenly split.

You are right on about some of the inherent self-selecting,and brilliant on how a conservative will answer a Fox poll while a liberal will hang up, and therefore Fox polls skew conservative (assuming that they identify themselves).

The area where polls are very useful is in measuring trends. Look for polls that use the precise same wording and questions over a period of time. They may have biases which render the actual measurement suspect (their 50 may be a 60 or a 40) but when they show a trend, up 10 from last week or down 5 or whatever, you can usually rely on that information.