Never trust the polls....NEVER!

[MrSoundAndVision]

Ok, I'm going to give a brief lesson in statistics (I have a degree in math), so here goes, this is why you should never trust a poll (a more formal report to come).

Let's say we have a certain poll which samples the choices of 100 democrats (I'm making this poll up, just so it's easy on me), and the pollsters tell you that John Kerry has 18% of the vote, with a margin of error of 5%. This naturally means that John Kerry has somewhere (anywhere) between 13% and 23% (big difference really, especially in this game). 5% is called the confidence interval. Here's what's left out, what you are not told about this poll:

-That statistical sampling (as this is a sampling, not every voter in the US is asked) is never considered absolutely accurate. Ok, that's obvious. But here is the zinger: they could just report how many respondents voted for John Kerry, but they didn't, they said that he had 18% + or - 5%. So, this means something. This means that they MAY HAVE sampled their sampling, and, in one sample, they found that JK had 18%. Strictly, they should say what they sampled.

-There is another number, we will call ALPHA. ALPHA is the confidence coefficient, and when you sample a population you would, in this case, say that "Our Pollster is ALPHA percent sure that these results are correct, that John Kerry would get somewhere between 13% and 23% of the vote if it were held today." So ALPHA is important. Some common ALPHA values are 5%. Applying above, this would mean that "We were 95% sure that JK...". So, ALPHA could be 20%. This would translate to "We were 80% sure..." and so on. To interpret, this means that there is a 20% chance we are entirely wrong. That in fact, we can't conculsively say that JK has this percentage. This is where ALPHA gets manipulated, THEY DON'T TELL YOU ALPHA. So when we have an ALPHA of 20%, and we don't get the results we want, we can resample, and have a good chance at getting the results we want, though they may be completely untrue.

That's it in short, I need to go through my Stats books and I'll get back to you formally. Basically you need to remember that statistics are incredibly imprecise, and incredibly easy to manipulate (just look at the insurance industry). Statistics dictates that you only have to have 30 samplings in order for them to be considered mathematically representative of the population (I know it sounds nuts, but it's true, look in any Stats book). So, even the most reputable polling company CAN CLAIM, without intent to mislead, that 30 voters here in Berea, KY represent all voters. And just imagine, if they don't get the results they want, just resample. A statistics programs (such as Excel) could resample with a single click. So watch those polls. Thanks.

[Feanorcurufinwe]

1. President Dewey would disagree

[JohnKleeb]

2. lol Ive never heard of President Dewey

[Mikimouse]

3. Bwhahahahahah! That'll teach 'em to do a telephone survey in 1948

I would love to know which rocket scientist designed that one! Gallup strikes again!

[robbedvoter]

13. Actually, Gallup gave up polling early in the race. "It's useless"

they said and many people believed Truman had no chance and voted third party (Wallace). And still didn't turn the tide!

[xray s]

4. The trend that really worries me...

...is news programs reporting internet polls without any explanation that they are completely without scientific merit and totally worthless.

[Peace_Place]

5. You are the first person I've heard say this

Ever since my college days (political science and economics major)I have marveled at polls and how people live and die by them but they never release the confidence level.

But then as I gained more experience I noticed the actual results were less than the polling numbers for the Dems. Being from NC I learned this lesson the hard way. Sen. Hunt, Sen. Gantt. The polls would always say Helms was going to lose and everytime they were way off.

It wasn't until a county Dem official told me the polls are purposefully skewed to give the false impression of Dem strength and Rep weakness. To encourage D's to vote and supress the R's. When I asked them about the 2000 Pres election, they said the VNS (voter news service) was releasing bad data to supress the vote. I would prefer Pres. Gore but I didn't enjoy the election aftermath that would have been avoided had the Central time zone of Fl. had voted.

Never believe the polls.

[MrSoundAndVision]

6. Thank you so much

you legitimized my report above. I am so glad you have heard confidence levels. So really any use of statistics should be questioned. Did you mean to imply that these polls were released (which showed high democrat winning percentages against Jesse Helms) to get Democrats to stay home? A little reverse-statistical-psychology aimed at the lazy American huh? I know plenty.

[Feanorcurufinwe]

7. Three kinds of lies:

lies, damned lies, and statistics.

I don't think you have to be a math major to figure out that a poll of 300 people can't predict how tens of millions of people will vote...but it doesn't hurt...

[MrSoundAndVision]

8. well thanks...

me neither, but some Deaners were propping up an ABC/Washington Post poll, and I thought an indirect response was necessary.

[Upfront]

9. Interesting

Question are you suggesting that ABC and the Wash. Post are skewing their poll? If so, why would they?

[Feanorcurufinwe]

10. What possible reason could the media have for deceiving us?

Isn't that kind of a far-fetched idea?

[JohnKleeb]

12. gasp

I never thought of that lol.

[Upfront]

15. Question Must Be To Hard

for some to grasp, so let try again. Why would anyone say that the ABC&Washington post poll would phony its results to give Dean a boost. I doubt they would, but then maybe that was not your intent, just asking. I do understand that this may happen, as I no longer believe in the tooth Fairy, as some, it seems do. Maybe there is good reason not to trust these polls when Dennis is polling less then unknown candidate. Looks funny to me. lol

[Feanorcurufinwe]

17. Are you seriously saying

that you can't think of any possible reason for the media to distort the news they report?

[JohnKleeb]

18. No I am talking about the media being deciteful in an overall sense

Cant deny that do that. I didnt accuse the post of nothin. I dont trust the polls ever, I didnt trust them when Bush was at his high point now did I. I dont trust the media really.

[helleborient]

19. In reality, much of the media are just a business like any other...

They follow the market and their particular niche...from liberal to conservative.

If we feel they are being unfair, form a relationship with a local reporter or editor and ask them about it.

I've done the same here, and most of our local newspaper people are just as overworked as everyone else. The newspaper industry is not a high profit margin industry, and there is a lot of pressure in the business.

Put your money and your eyes on the media you like best and convince others to do so...the market will head your direction.

[DemBones DemBones]

14. Most people never take statistics because

it's >ack!< MATH!!! Math-phobia is a serious problem in this country. I blame a lot of that on teachers who only show one way to work a problem and blow off the kids who don't get it. Too many elementary school teachers are weak in math and science. The schools would benefit from haaving math and science specialists to teach those subjects.

[helleborient]

24. And it can be fun...Really, I"m serious...

I spent a lot of time in college tutoring math and trying to come up with innovative, interesting ways to teach it. Plus, I had an outstanding statistics prof!

Anyone who says they would never use statistics in the future is woefully uninformed. And that's why, unfortunately, they get taken in.

[JohnKleeb]

25. ewwwwwww math my nemesis

:puke: get it away from me.

[Mairead]

29. in soc and psych, the minimum standard for confidence is p=.05

i.e. 95% confidence.

In my experience, the question itself determines the answers you get in a way that even the most impeccable sampling can't overcome. If you ask a question that has ambiguity built in, and you fail to sort out that ambiguity in subsequent questions, then no matter how large and how random the sample, the answers are going to be something approaching rubbish.

[robbedvoter]

11. I only believe in trends. Same sample in time - changes are significant/

Numbers per se - are worthless, even without lying.
But after visiting pollkatz, I believe in time variations.

[helleborient]

16. What some polls do report...

Some polls do report the confidence level, and when they do it is usually standard that the margin of error refers to a confidence level of 95% - in other words a 5% chance that the poll is totally out of wack beyond the margin of error.

It is impressive, in my mind, that the use of a relatively small sample is as robust statistically as it is.

This use of statistical confidence is used in nearly every aspect of our lives.

When I moved to Michigan and took the driver's exam...I answered 7 questions and was deemed to have passed, because the computer testing involved giving you questions that statistically weeded out those who would pass from those who did not.

Statistical levels of confidence dominate the world of scientific research.

Basically, statistics is the only method we have to attempt to predict anything.

If you are not interested in having an idea what might be happening in a campaign before the actual voting date, then don't read polls at all.

If you want some kind of picture of the way things are heading - most polls conducted by reputable polling firms are reasonably statistically robust.

Saying they mean absolutely nothing is a lot like throwing out penicillin as a drug because it doesn't make some people well.

[Feanorcurufinwe]

20. Have faith in the media then. Believe what you are told.

If the polls say your candidate will lose, you might as well stay home on election day as well.

By the way, could you cite a poll that DOES report the confidence level as well as the MOE?

[helleborient]

21. No, I think everyone should take a statistics course in college...

And learn what a poll actually means. It's a snapshot in time, and certainly no reason to stay home from the polls.

Here's your link:

http://www.theiowachannel.com/politics/2597550/detail.html

"The Research 2000 Iowa Poll was conducted from Oct. 26 through Oct. 28, 2003.

Those interviewed were selected by the random variation of the last four digits of telephone numbers. A cross-section of exchanges were utilized in order to ensure an accurate reflection of the state. Quotas were assigned to reflect the voter registration of distribution by county.

The margin for error, according to standards customarily used by statisticians, is no more than plus or minus 4 percentage points. This means that there is a 95 percent probability that the true figure would fall within that range if the entire population were sampled. The margin for error is higher for any subgroup, such as for gender or party affiliation.

There was an oversampling conducted among Democratic caucus voters. The margin for error is plus or minus 5%."

These people are great...the other day when someone wondered why they didn't report the entire results of a poll, I sent an e-mail and asked, and it was up within an hour with a very kind, polite reply to my e-mail. They put some of the text up first and within a few hours had the complete details.

Media people are first and foremost people...which means they have beliefs, principles, etc....like everyone else. If you treat them life a faceless entity, you get a faceless entity back. If you treat them like people, you might even sway coverage!

[Feanorcurufinwe]

22. So where's the confidence level?

all they report is the MOE.

[helleborient]

23. Here it is...

I'll quote again:

"there is a 95 percent probability that the true figure would fall within that range if the entire population were sampled."

That's a 95-percent confidence interval.

[MrSoundAndVision]

28. Ok uh

I knew I should have looked up my s**t before I posted the above. First: You're right, for terminological purposes, statisticians do say they have a 95% confidence interval.

Ok, here's my problem, maybe you can help me, because it's been a while since I had stats: In my example we say that John Kerry gets 18% of the vote with a margin of error of 5%. So that means that the true vote of the population would yield somewhere between 13% and 23% for John Kerry. But that just tells you the size of the inverval. But, that doesn't tell you how often this figure is wrong. Where is that figure and what is it called?

So if we lessen this interval to, say, 3%, then that doesn't mean that we have a 3% probability of having a misrepresentative result (I never should have said I'm a mathematician, makes me look like some failing authority, but it's been a while!).

[dsc]

31. You are mixing up two things

The MOE is what determines the size of the interval. The confidence level is how often it is wrong. In your example the first interval would be wrong, on average, 5% of the time. In the second, it would also be wrong 5% of the time if the confidence level is the same.

[dsc]

26. Several things

First, Confidence intervals are assumed to be with 95% confidence unless otherwise stated. That is standard for all polls.

Second, I have literally never seen the idea presented that only 30 people can be a valid sample of a population. Virtually every poll I have ever seen has stated the number of people asked. Usually in the 300 to 500 range depending on the state and the demographic breakdown of the poll. I have literally never, ever, seen a poll of less than 300 and have seen them with up to 1500.

Third, Ethical pollsters don't repoll and samples are assumed to be randomly chosen or else the rest of the stuff doesn't work. Pollsters can't pick people from some town in KY to represent everyone with no intent to mislead. If one intends to represent all of Kentucky one most randomly pick people from all of Kentucky. Not doing so is the defintion of an intent to mislead.

[MrSoundAndVision]

27. Right I said that...

Ok so you say polls are done with 95% confidence unless stated otherwise. Thanks, I didn't know that, but the real point that I wanted to make was that even with that confidence level all one must do is resample to get the desired result.

Second, actually statistics does teach that one only needs to have 30 samples to determine a distibution. I know that 30 people are not representative (that's why I emphasized that it sounds nuts). And personally, I don't think 300 people would be representative of this country.

Thirdly, the basic point I wanted to make was that pollsters can't be trusted, the temptation being to great and the means to easy to manipulate the data.

So here's a question, how far does a pollster have to go to ethically obtain a random distribution? I mean, if you poll by telephone you're leaving out people without telephones? Or are you ethical if you poll people who visit the CNN website?

I think the important thing is that people understand how easy sampling can be manipulated and still be presented as indicative. I mean, if the standard sampling is with 95%, then people should know that one in twenty polls they see are wrong EVEN in the most ethical case, where sampling is completely random. This post is not about attacking reputable organizations, or about claiming that they only sample 30 people, the standards of polls in the 21st Century: it's about disseminating a little understanding of stats.

Never trust the polls, at least, don't trust them to assist in your decision-making process when selecting a presidential candidate, or deciding which one is viable. Thus http://www.kucinich.us

Thanks

[helleborient]

30. Please please please...learn more about statistics!

Probably the most relevant thing you said was that 1 out of 20 polls is likely to be off base with a 95% confidence interval...so if you read 6 different polls and they all say basically the same thing, or just 1 seems off - what the 5 say is most likely a pretty accurate representation of reality.

Never never depend on one poll to give you the picture of reality - I agree with that...but compare multiple ones.

We have no shortage of polls about the Democratic nominating process...so definitely look at mulitple ones!

[MrSoundAndVision]

32. See I knew I could trust people to get all hell bent on this

I mean, I said it was informal, I said a better report would come later. It wasn't meant to be an authoritative document, I said that. Anyway, I think ethical sampling is relevent; I think it's relevent that it is so easy to manipulate the sampling process; I don't think that we can trust media organizations or those with corporate interest in the election process to do our polling, these are the same organizations that marched us into war based on lies and deceit, a war your candidate opposed so much he led an effort to stop it, oh wait that was somebody else. Processes of manipulation in polling can be applied to more than one poll.

So the media likes your candidate, thus you trust them. I don't trust them, because they're trying to take Dennis Kucinich out of the Democratic process because of a corporate conflict of interest. I guess that's what boils down too. You're not willing to see these methods because the media has so obviously chosen your candidate for you.