If you see "study says X", but your personal experience is Y, do you reflexively yell "bullshit!"?

[Silent3]

It might be true that a particular study is bullshit, but I get the impression that way, way too many people immediately interpret any report of a scientific study as if the study itself is claiming to be an iron-clad proclamation about how some food or drug or experience does or does not work for every single person on the planet, and therefore if you know of any exceptions at all to this instantly-formed straw man version of the study, then the study itself is bullshit.

First of all, popular press reports of many (most even?) scientific studies are quite often sensationalized and oversimplified. Unless you have bothered to look up the actually original study, it is safe to assume the actual study is much more modest and narrow in its findings than clickbait headlines might lead you to believe.

Second, you might have to contend with your own internal oversimplification process. A lot of people seem to want to round 49% down to 0%, and 51% up to 100%.

If you say "most people like chocolate", it will be generally understood that you realize there are people who don't like chocolate, and few people are going to scream "Bullshit!" at you simply because they personally don't like chocolate.

Replace "chocolate" with "examplenine" and "like" with "effective at reducing headaches", and suddenly half the population are idiots who can't grasp the simple difference between "most" and "all", and are really pissed at you that you dared to say something that didn't explicitly validate their personal experience.

[Evergreen Emerald]

1. I only believe in science if I agree with it.

[SoonerPride]

2. "studies" can be very good science or complete nonsense (VIDEO)

I take any news headline regarding studies with a grain of salt.

If it is from a scientific journal with strict control groups and sound methodology then it holds weight.

But most "studies" you see in the media are junk.

John Oliver did a deep dive on this topic. Worth the watch:

[shrike3]

3. John Oliver's a treasure.

[SoonerPride]

5. Yes he is!

[Silent3]

11. The big problem is the reflexive "bullshit!" reaction is practically instantaneous, faster than...

...any consideration of whether a particular study lurking behind a popular press account is any good or not. It's not a matter of whether the there's good reason or not good reason to doubt a particular study (as I said before, there may well be).

I saw the Last Week Tonight thing, which was excellent, but it is a bit besides the point to what I'm talking about, except for how the garbage that John Oliver is talking about might contribute to a general lack of trust in any scientific studies, well done or not.

The problem I'm specifically addressing is the excessive importance people assign to their own personal experiences, combined with a situational blindness to important and not-so-subtle distinctions, like between "most" and "all".

[SoonerPride]

13. I undertand what you are saying. People feel their personal experience trumps data.

It is an inherent bias that is hard to overcome.

[Johnny2X2X]

4. Scientific literacy is waning in the US

You hit the nail on the head.

And it's always disappointing for me to come to DU and see so many people arguing against a study because it goes against their personal beliefs so they then makeup excuses and make false arguments. "Oh, well I've been using toe nail clippings to treat my heart burn for years, and it works great!" and "Big Pharma is anti toe nail clippings and funded this study!!!"

And also, 1 study is just 1 study, it's a preponderance of studies that determine paths forward to policy and health issues.

[shrike3]

6. "Scientific literacy is waning."

An understatement.

[Pobeka]

9. I'll somewhat disagree. True science reporters in news organizations are a big problem.

But there appears to be a boatload of people getting high school diplomas without any concept at all about fundamental science principles.

See my post #7.

[SoonerPride]

15. I'm not sure we ever had much scientific literacy as a society.

..........or a planet.

But critical thinking is certainly on the decline, in so much as what few percentage of people engaged in it to begin with.

[Pobeka]

7. Worst problem is "science" reporters failing to read and understand the actual study.

I was a scientist for 31 years, and you know what, most studies and papers take 4 or 5 hours to really read and understand.

I see so much bad reporting -- articles written claiming "study says X", when sometimes literally the next paragraph in the report says "X is an assumption that almost certainly will not hold up".

So the first stupid "news report" comes out, and then that report is repeated over and over, because news organizations don't have the a) the skilled staff with b) the time to read a study paper completely and understand it.

[Jilly_in_VA]

8. It depends on who does the "study"

Some of them are so biased they are worthless, and all of you should know this. As my Stix instructor said at the very beginning of the semester, "You can make statistics prove anything. What I am going to try to teach you is how to make those statistics valid."

[Silent3]

14. Yes, it does depend on the quality of the study. But if you instantly decide the quality...

...of a study must be bad because the study doesn't match your personal experience, then you have a scientific literacy problem, regardless of whether or not the study turns out to be well done or not.

[MissMillie]

10. I've been an outlier before, so I know that just because some things are not likely

it does not necessarily mean that they are impossible.

I was the 13 year-old with shingles, initially misdiagnosed because "13 year-olds don't get shingles."

And my kid never cared for chocolate

[Silent3]

12. Does that mean you should react to a study that says...

...something is unlikely as if the study said it was impossible, and be angry about that, and call it bullshit?

[MissMillie]

19. I think it means

"unlikely" means "unlikely" and "impossible" means "impossible." LOL

of course other qualifying words come into play, such as "highly unlikely" or "extremely unlikely," or even "nearly impossible." And when they give actual numbers and percentages, that needs to be considered as well.

I have experience with research papers--in the past I worked for research faculty. So while I'm not a scientist/mathematician, I have been involved with preparing academic papers--for peer review.

I have no doubt that sometimes the press will look for a good headline where there really isn't one, and I also believe that the public often accepts the headlines without reading (or understanding) the details. (And I say that without having any data to back that up.)

[treestar]

16. I would not. My personal experience is anecdotal

Thus I put no stock in "everyone I know" doesn't apply, as everyone a person knows is not a scientific sample.

[keithbvadu2]

17. Research pays off

[sanatanadharma]

18. I know that every 'thing' I know is only an approximation of true reality

I know that everything I know, the world in my perception, is only an approximation of true reality.
Humility is accepting that I may be wrong about what I perceive. Wisdom is accepting that truth trumps all perceptions.
Humanity rests in accepting others without demanding they change to match my perception (desire-belief-system).

Water is real. I can see, feel, taste (what is the taste of?) water.
But water, science says, is actually H2O molecules which are actually hydrogen and oxygen atoms which are electrons and other things made of other realities that I can not see, taste, touch, smell, hear, but only conceptualize. And then more concepts are needed to explain the recent explanation. Wherein is truth located?

In each individual there exists the belief that what one says "I know" is true. But in some individuals the desire to be 'right' destroys the ability to be corrected, to be correct. Only those who own-up to not-knowing are qualified to learn.

Be the one who accepts desires fulfilled (knowing) but cherishes those desires denied (not-knowing), for there is where your freedom lies.
Or so I would tell those who "immediately interpret any report of a scientific study as if the study itself is claiming to be an iron-clad proclamation".

True story: I was applying for a management job at a chocolate business; no relevant previous experience. When the GM introduced me to the owner, the owner asked me if I liked chocolate. I said, "I can take it or leave it.)
Twenty years later, at my retirement party, the owner brought that moment up to all.

[Torchlight]

20. I give research a greater benefit of a doubt

than I do anecdotal data.

Every time I hear someone say to the effect "I didn't wear a seat belt when I was a kid and was never hurt" or "I was beaten regularly when I was growing up and turned out fine" it tends to reinforce my position.

Granted, individuals are sometimes honest and research is sometimes flawed.

[yonder]

21. A good post, S3. Hats off to you as well as the many excellent comments.