IQ tests, their limits, and why do they use a normal distribution as the common 'score'?

[muriel_volestrangler]

The thread in R&T about IQ scores and atheism has attracted a few IQ claims, and sceptical replies from those who know what an IQ of 220 would actually mean - Gentle Giant helpfully posted a link to this table, which shows that (with the IQ distribution being a normal distribution around 100, with 15 as 1 standard deviation) 170 is roughly 1 in a million, and 220 1 in 1.6 quadrillion (for comparison, one estimate of all Homo sapiens that have ever lived is 100 billion, which I reckon would mean the most intelligent person ever had an IQ of about 200; the most intelligent person alive might be expected to have about 195. There is, of course, a possible aspect of "well, it's just chance that the most intelligent human ever, in the past or the future, is posting on DU at the moment").

It was also posted that IQ tests can't really measure an IQ above about 150-155 - this sounds believable to me, because that would be about 3.5 standard deviations, or 1 in 5000, and I doubt you could devise tests that could pick out the 1 in 5000 at all accurately. But that's just me and my innate scepticism, rather than any knowledge or references. Can anyone say if that is a generally held limit, among those who do think IQ tests show something measurable?

And that last part shows the question I was begging - is the concept of IQ meaningful anyway?

Finally, wouldn't it be a lot more useful, both to weed out claims that effectively say "I'm one of the 300 most intelligent people in the USA", and in the more everyday world, for easier public understanding, to say "X is in the top 2% of intelligence" or "Y is in the top 70%" (or "bottom 30%" - discuss), because that relates to numbers and proportions that we can all think about, rather than "IQ of 95"? What's the use of a manufactured normal distribution?

[salvorhardin]

1. I'm not sure I understand your last question

The distribution isn't manufactured. Mental abilities, no matter how one measures them, do tend to fall on a normal distribution, more or less (it's not quite a normal distribution because there tend to be more people at either of the extremes than there would in a perfectly normal distribution).

[muriel_volestrangler]

2. But the definition of IQ is that it's a normal distribution

How do you actually quantify mental abilities? You can say, for a particular test, 'how many questions does a person get right?' But where the mode falls is going to depend on how difficult the test is. So any test score has to be converted, presumably after giving it to a decent-sized sample, to a normal distribution.

eg:

currently IQ is calculated by estimating where, under the normal distribution curve, someone’s performance on an IQ test places him or her. The curve is standardized such that the mean score is 100 and the standard deviation around the mean is 15. An average IQ is therefore between 85 and 115. The scale will go from 55 to 145 (three standard deviations below and above the mean).

Principles of Organizational Behaviour 4e: Glossary

(interesting that implies they would both quoting a figure above 145).

The 'more people at either extreme' shouldn't be true at all, because it's defined as being a normal distribution, with nothing about "except for the extremes".

It seems to me that, for each test devised, they have to find out what the top 1% score, top 5%, and so on, and then they use that information to convert the test score into a normal distribution. In effect, they're pretending that there is an ideal test, on which the mean score is 100, and so on. But since that test doesn't actually exist, telling people what they would get on it seem less useful than saying "you're in the 60th percentile".

[varkam]

3. A few thoughts

I'm fortunate enough that I actually have a little bit of training in IQ tests. At one point, I was giving IQ tests such as the Wechsler series in graduate school as well as scoring them and interpreting the results. The thing that most people don't realize about IQ tests is that they are just one piece of data. More than anything, they are a tool. Usually they are used in conjunction with other tools, such as observation and interviews, in order to try to get a complete picture about the cognitive abilities that a person has. They do not, as some seem to think, accurately capture the total cognitive ability of an individual. There are many kinds of intelligence, most of which aren't measured on such tests.

That is not to say, however, that IQ tests aren't measuring something that is actually there. I think that IQ tests do tap into intelligence given that they are administered correctly (which means a lot of different things and is a big assumption). Could there be more meaningful ways of conceptualizing and measuring human intelligence? Absolutely. I think that as the science of psychology becomes more of a, well, science, we're going to see methods of measuring intelligence that don't rely upon cultural assumptions - which is one of the main criticisms of IQ tests in general.

I do not think, however, that the standard distribution is manufactured. A heaping ton of normative data has been collected using these tests, and it has been found that the population does, indeed, fall along a normal distribution and that an IQ test is one way of tapping into that. To an extent, the standard distribution is just an assumption as it is impossible to give the test to every man, woman, and child. It is a useful assumption though, and one that is theoretically sound and has been borne out by the data. So, again, I wouldn't say that the distribution is manufactured - though I might not be understanding the question.

Oh, and I'm not a doc or anything - so take what I have to say with a grain of salt :hi:

[LeftishBrit]

4. The point about it being hard to measure an IQ over 150 is very valid

Once you get beyond three standard deviations from the mean (most IQ tests nowadays treat one standard deviation as 15 points), measurement becomes much more unreliable. Also, it becomes harder to tell what it means. In general, we know for example that someone with an IQ of 120 is more likely to get into university than someone with an IQ of 100. But is someone with an IQ of 170 more likely to make an earthshaking scientific discovery, or produce a great work of art, than someone with an IQ of 150? Insofar as one can get such evidence, the answer seems to be No. IQ is correlated with creative achievement up to about a score of 120. However, beyond that point for the arts, or at most 130 for the sciences, the correlation disappears, and other factors than IQ become crucial.

Does IQ mean anything? Huge debate on this. IQ certainly *correlates* with lots of things, especially academic achievement. And someone with a very low IQ is likely to have a lot of problems both in school learning and social functioning. However, what IQ tests actually measure, and whether it should be called 'intelligence', and whether it's even a single entity, are all much more debatable issues. Many people think that intelligence should be seen in terms of multiple correlated components, rather than one single factor. I do; though I think that Howard Gardner's version of multiple-intelligence theory is flawed and simplistic. Others do think that IQ can be reduced to a single over-arching factor.

Another problem is that IQ tests were designed for use in school-oriented cultures; and may not be valid in other cultures. Apart from cultural differences in knowledge of specific content of tests, the whole concept of being asked questions to test your knowledge, rather than for a practical purpose, or of having to reason about hypothetical situations, may be very unfamiliar to those who haven't been to school. Michael Cole and Sylvia Scribner did a lot of work on this subject in the 1970s and 80s.

[Random_Australian]

5. My input shall be about the normal distribution thing then, as others have yakked about the other

stuff.

It's not actually a manufactured normal distribution (and there are actually two perturbations from the normal distribution).

You administer a test to a cohort, and the results (when large enough quantities of people are used) will fall into a distribution of its own.

What does this look like? Well, there is an interesting mathematical theorem (Can't remember the name, maybe Central Limit Theorem?) That says if measure something that is the sum of many variables with any distribution (even, say, a series of exponential decays) then the distribution of the sum is a normal distribution.

Well, almost. There are a couple of qualifications that haven't been said, one of which is the error in testing which leads to larger 'wings', and the fact that this is only true when intelligence is determined by the sum of many variables, none of which are far more important than the others.

Well, in the case of brain damage, that happens to be far more important than the other variables, and leads to a 'bump' at low IQ that is not part of a normal distribution.

(Interesting aside: The concept of many small variables going to a normal distribution is quite general. It governs things like height, weight, length of manufactured rods, et cetera)

But that said, if you have done your test well, you will get the same distribution every time. Due to the way you score your test, it will be stretched and might be centered wherever. (For instance, in one test, you might award negative points for wrong answers, which would move the center, or you might have 10 times as many questions on a test, and thus get a lot longer distribution).

Because the scale and center are arbitrary, to compare it, you put it on the same scale and center as the other tests - mean 100, SD 15.

In other words, you don't lose any information by using mean 100 SD 15.

[salvorhardin]

7. Is it testing errors that account for the bumps in the wings?

It's been a long time since my Psychological Testing course but I was under the impression that the bumps at the extremes weren't readily explainable, much like we're not really sure what's causing the Flynn effect (although there are some good hypotheses).

[Random_Australian]

8. There is probably more going on, sure.

In fact, correlation between variables would also do it, and I would expect that no-one actually knows where it comes from.

I was more listing a source of error, which is what you get from generating a score from a discrete number of questions. (In other words, since you can never get a half, you get an average of a half, which has more in the wings than the real distribution)

The way I think means that is the very first thing my mind leapt to. :)

And to clarify, the bump at low IQ is considered to be caused as I have laid out.

[Random_Australian]

6. Is the concept of IQ meaningful?

Sometimes, yes.

It is fairly well documented that a person's score in any one test correlates with with their score in any other test, with R^2 going from ~0.3 to 0.8, so by giving someone a battery of tests, you can be reasonably confident that their score will correlate with their ability to perform in any other test.

In other words, you have a measure that statistically correlates with intelligence, so it would be useful in looking for selection effects - will hairdressers show the same IQ distribution as the rest of the population?
(Note: I am *not* going into the debate about intelligence)