Here is why you should take nutritional studies with a grain of salt

judywoody

I think we knew it all along but the details are pretty shocking - here is a guy who deliberately set up a legit research study to fool the press into reporting that you have to eat chocolate everyday to lose weight:

http://io9.com/i-fooled-millions-into-thinking-chocolate-helps-weight-1707251800

ultrahoon

This isn't a reason to ignore what a study says, it's a damn good reason to check their sources instead. Over 50% of the journals took this article on, and if they'd done even a tiny amount of fact checking, that figure would have been 0% (the doctor doesn't actually exist, the institute doesn't actually exist, the sample size is well below any sane threshold etc)

Not all scientific studies are created equal, but there's no reason to blanket apply the 'grain of salt', one can just actually read the paper itself instead of someone elses summary of it, and get to the bottom of the issue.

judywoody

ultrahoon wrote: »

This isn't a reason to ignore what a study says, it's a damn good reason to check their sources instead. Over 50% of the journals took this article on, and if they'd done even a tiny amount of fact checking, that figure would have been 0% (the doctor doesn't actually exist, the institute doesn't actually exist, the sample size is well below any sane threshold etc)

Not all scientific studies are created equal, but there's no reason to blanket apply the 'grain of salt', one can just actually read the paper itself instead of someone elses summary of it, and get to the bottom of the issue.

That's true but remember that many studies quoted by those newspapers were exactly done that way. Should journalists be less lazy? Of course they should. Bear in mind though that journalists are not Scientists either. So the root problem is that there are still Scientists who conduct studies I this manner and the layman is not always able to interpret them correctly. And the results are often repeated by armchair nutritionists.

LKArgh

judywoody wrote: »

I think we knew it all along but the details are pretty shocking - here is a guy who deliberately set up a legit research study to fool the press into reporting that you have to eat chocolate everyday to lose weight:

http://io9.com/i-fooled-millions-into-thinking-chocolate-helps-weight-1707251800

Newspapers are not reliable sources of information when it comes to scientific research. Which is exactly what this man proved. But fooling Cosmopolitan and the Daily Star is not the same as fooling an actual scientific journal. This journalist has proved that if you want to find out scientific research and facts, you should look into peer reviewed journals or at least reputable websites (e.g. universities, research institutes or medical associations) not look for science next to tips about how to spice up your sex life and gossip about the sexual adventures of reality show celebrities

stevencloser

That's not why you should take studies with a grain of salt, but news articles about studies.

ultrahoon

judywoody wrote: »

ultrahoon wrote: »

This isn't a reason to ignore what a study says, it's a damn good reason to check their sources instead. Over 50% of the journals took this article on, and if they'd done even a tiny amount of fact checking, that figure would have been 0% (the doctor doesn't actually exist, the institute doesn't actually exist, the sample size is well below any sane threshold etc)

Not all scientific studies are created equal, but there's no reason to blanket apply the 'grain of salt', one can just actually read the paper itself instead of someone elses summary of it, and get to the bottom of the issue.

That's true but remember that many studies quoted by those newspapers were exactly done that way. Should journalists be less lazy? Of course they should. Bear in mind though that journalists are not Scientists either. So the root problem is that there are still Scientists who conduct studies I this manner and the layman is not always able to interpret them correctly. And the results are often repeated by armchair nutritionists.

No, the root problem is that there are people who will blindly accept / reject a summary of something without looking the raw information because it fits with a narrative they either want to be true, or are afraid might be true. Peer reviewed journals are a great way to weed out the bad science, that's why we use them. This article may have been accepted by the journals, but it got completely and utterly eviscerated by the peers reviewing it.

You don't need to understand anything at all about how fat is burned to note the number of participants in a study, of the lack of a control test.

Pawsforme

Agreed -- that article says nothing at all about the validity of peer reviewed studies published in major scientific journals. It does say quite a lot about the media.

rileysowner

As someone who has a good friend who does scientific research as his daily job, I learned to not trust the general media's understanding of scientific studies or their validity long ago. I cannot remember what the study was on, it was not nutrition but it was human biology, and the papers and TV news presented the findings in a way that was completely wrong, almost opposite of the actual conclusions of the study. I asked him about this study that got a lot of press at the time, and he made me aware of how the media's take on it was completely wrong. Since then, I pretty much ignore what the media has to say about things except in so far as looking up the actual study if it looks interesting. Face, newspapers, magazines, television news, and the rest all need to sell their product. A headline about chocolate being necessary for weight loss would do that. Perhaps in the past they might have, note might have, looked more deeply; but today, competition is such that doing due diligence for the media is far less likely.

judywoody

You are obviously correct that many newspapers just report the exact opposite of the findings. However, this article shows that even bad science studies where newspapers report the results exactly the same way they were told, are often not peer reviewed and to be honest the ultimate fault lies with the study (as the author of this article admits)

Either way, not many people know what makes a good study and what makes a bad study. To expect them to know that is wishful thinking. I am by no means questioning all scientific studies just that there are also many bad studies (not just bad newspapers).

I think bad studies shouldnt exist in the first place and scienctific studies and their interpretation should be made available in laymens language - when I read the original research paper I fall asleep somewhere between the first and second paragraph (I am sure many would agree). Not everyone studies biology or nutrition at university nor should they be expected to.