Pseudoscience and bad advice

lulabox

I'm a fairly new user of MFP and I have noticed that there is a lot of pseudoscience floating around on some threads on metabolism/medical topics, ranging from the plain ridiculous to the slightly dubious. For non specialists or people who haven't had some kind of postgraduate scientific training, it must be difficult to assess the validity of a source of information.

As a rule of thumb, I would recommend:

- Trusting national (non-commercial) health bodies and not-for-profit medical charities (and I do mean medical - no holistic or alternative medicine sources): NHS, British Heat Foundation, Cancer Research etc.
- Trusting health advice posted on reputable university websites or medical websites: Harvard Public Health, Columbia Medical school etc.
- Your doctor.

I wouldn't trust:

- Health, fitness or other types of magazines for scientific or nutrition advice. A lot of it is non-sense and influenced by advertisers.
- Self proclaimed experts online, whether a website (e.g. Patric Holford) or a person on a forum.
- Anyone who is very aggressive with their points. Anybody with a good argument should be able to carefully debunk someone else's and provide enough information for the reader to make their own mind about it.
- Unless you're an expert in that particular field, peer reviewed literature (=published science articles on Pubmed or elsewhere).
- Any celebrity trainer or nutritionist without really looking into it and asking my doctor for advice.

Research papers are a bunch of hypotheses with data to support or reject them. Often their findings are taken up by the press or individuals but these people haven't truly appreciated the whole picture: a nutrient from broccoli tested in vitro in a lab with good results for cancer doesn't mean it actually helps with cancer in real people.

Without reading the paper fully and actually understanding the statistics behind it, you can't quote abstract without appreciating the study design: sample number, control group, statistical methods etc. It's actually a steep learning curve and I hope I will master critical appraisal of research in my field in the next couple of years as I go through my PhD. People on forums are for the majority amateurs who have a tendency to cherry pick sources of information to validate their claims ("quackery" it's called by some).

Only expert in the field will probably be able to assess the validity of the claim and the peer review is less than foolproof. Bad papers get into good journals all the time, and there are lots of bad journals out there. Also, always look at competing interested, who funded the study (public funds, private body, pharma etc) and where it was done.

I recommend the following books:
Bad Science http://www.amazon.co.uk/Bad-Science-Ben-Goldacre/dp/0007240198http://www.amazon.co.uk/Bad-Science-Ben-Goldacre/dp/0007240198
Hoe to lie with statistics http://www.amazon.co.uk/How-Lie-Statistics-Penguin-Business/dp/0140136290/ref=pd_sim_b_21

Oh and obviously I am a random on the internet so make what you will of this advice...

LabRat529

Great post. Kinda got lost in the flow, so I'm bumping it up. I'm not entirely sure I agree with all your points... perhaps I over-estimate human intelligence (or misunderstand what you are saying), but I'd much rather have someone digging through peer-reviewed literature despite being an novice in the field. You've got to go somewhere for the truth, right? Of course, you don't want to take one single paper and build your life around it. You need to critically analyze multiple papers and draw conclusions that way. And of course you need to be aware that science evolves. Our understanding of 'the truth' changes as we collect more evidence for or against a specific hypothesis.

Also... a bit of advice for you as you chase down that elusive PhD... even the 'experts' in the field have their biases. And man oh man does it get ugly when you put two or more different 'experts' on the same topic in the room and let them have at each other! Scientist sure love to fight *cough* debate... I mean debate respectfully. Yeah.

Good luck with your education! What field are you studying?

lulabox

I think that if a novice want to personally dig into peer reviewed literature, that's fantastic. I'm a big supporter of open access for that reason. I regularly dig into new fields out of interest or curiosity. The problem is just when a novice who begins to read the literature then starts giving advice - it should be taken with the caveat that this person may not fully understand or appreciate the material. Understanding and assessing medical statistics isn't particularly easy,

Bias and researchers: Absolutely hence me recommending "how to lie with statistics". My biggest moment I think was some time ago when I started being able to tear apart papers from so called eminent people in the field.

I see ego driven bashing between scientists all the time. Drives me nuts. I was talking to the wife of an extremely eminent scientist now in his 80s the other day, and she was saying that her and her husband have come to the conclusion that the ego bashing and the aggressiveness ultimately is what drives a lot of women out of science because it's what shapes the scientific working environment and culture with all its pressures. Interesting theory.

I am a biochemist turned biologist/marine geneticist. I have a wide range of interest, from population genetics (my PhD and area of expertise, or at least it will be one day!) to metabolism (something I got into during my biochemistry degree, but I'm not an expert).

becoming_a_new_me

That is why the bottom of the page says "Posts by members, moderators and admins should not be considered medical advice and no guarantee is made against accuracy."

lulabox

That is why the bottom of the page says "Posts by members, moderators and admins should not be considered medical advice and no guarantee is made against accuracy."

You know what I never noticed... Absolutely and same goes outside of MFP where no such kind reminder is visible.

nikolaim5

Welcome to the internet, nice to have you.

SteveTries

Good post and very good advice.

I'd like to emphasise the point on research papers. I've seen quite a lot of people on the forums refer to the abstract of a research paper and proffer that up as advice. But there are even more people who listen to that first guys posting and then go off quoting HIM as an authority. Madness.

Good example this week is a reference to a paper where the abstract concludes that Low GI or High GI diets are no different for weight loss. If you take an hour to read the paper you realise that the test was in no way relevant to the fat loss programmes people are undertaking in the real world - very briefly it was measuring the fat loss 9 months after cessation of dieting, oh and the diet itself was in the 100-300 caloric defecit range. Amusingly it dismissed the fact that the low GI group lost 50% more weight as statistically irrelevant.

Which brings me on to my second point. Whilst I agree with you about trusted sources, something you have to bear in mind is that those sources actually take a very long time to adopt new practices. With good reason in many cases but it's still typically well over 10 years for evidence based medical practice to be adopted.

So I would augment your excellent post with the statement that it's not a bad thing to consider other sources so long as you review it thoroughly and draw your own conclusions.

granolagrl85

Hmmm...ppl should think for themselves. I'm a nurse and still don't follow most NIH, CDC, Mayo Clinic rules. Give me some butter and bacon and take your grains.

Collinsky

I'm not so sure I'd trust a regular MD for all manner of nutritional or exercise physiology advice. They also have to be double checked against available, reliable research... unless my dr has taken additional training in nutrition, etc, then I'd rather trust an expert *in that field*.

lulabox

Good post and very good advice.

I'd like to emphasise the point on research papers. I've seen quite a lot of people on the forums refer to the abstract of a research paper and proffer that up as advice. But there are even more people who listen to that first guys posting and then go off quoting HIM as an authority. Madness.

Good example this week is a reference to a paper where the abstract concludes that Low GI or High GI diets are no different for weight loss. If you take an hour to read the paper you realise that the test was in no way relevant to the fat loss programmes people are undertaking in the real world - very briefly it was measuring the fat loss 9 months after cessation of dieting, oh and the diet itself was in the 100-300 caloric defecit range. Amusingly it dismissed the fact that the low GI group lost 50% more weight as statistically irrelevant.

Which brings me on to my second point. Whilst I agree with you about trusted sources, something you have to bear in mind is that those sources actually take a very long time to adopt new practices. With good reason in many cases but it's still typically well over 10 years for evidence based medical practice to be adopted.

So I would augment your excellent post with the statement that it's not a bad thing to consider other sources so long as you review it thoroughly and draw your own conclusions.

Absolutely. I think your last sentence is spot on and I wholeheartedly agree with your points.

lulabox

I'm not so sure I'd trust a regular MD for all manner of nutritional or exercise physiology advice. They also have to be double checked against available, reliable research... unless my dr has taken additional training in nutrition, etc, then I'd rather trust an expert *in that field*.

For sure if you can access an expert in the field. If not, I'd rather trust them over random info on the web.

lulabox

Hmmm...ppl should think for themselves. I'm a nurse and still don't follow most NIH, CDC, Mayo Clinic rules. Give me some butter and bacon and take your grains.

Hehehe I agree - eggs with yolks with that too. I think these general sites are good for people who are clueless or starting out - it's not harmful or plain quackery and better than randoms on the internet.

meerkat70

OP, are you quite serious? You'd trust a doctor (typically someone with little or no training on nutrition and fitness, and with a busy practice which tends to mean they can only digest drug-company-managed medical information to keep 'up to date') over peer reviewed journal based information?

On what grounds?

dj_stevie_c

In conclusion.... Research, find what works with you and GO FOR IT!



And don't be nasty to others who may have found another way that works for them Share don't scare!

NKF92879

Excellent post. I'm not trying to offend or me mean, but I will ALWAYS take the advice of a medical professional over the advice from someone online.

missy_1975

Yes, those dreadful holistic people know nothing...:huh: Excuse me whilst I go sell someone some snake oil

lulabox

OP, are you quite serious? You'd trust a doctor (typically someone with little or no training on nutrition and fitness, and with a busy practice which tends to mean they can only digest drug-company-managed medical information to keep 'up to date') over peer reviewed journal based information?

On what grounds?

I would trust a GP in the UK with basic health and medical advice with regards to conditions that are linked to nutrition. High cholesterol, obesity, cardiovascular disease and the likes are very commonly encountered by GPs nowadays and most of them are pretty clued up with national guidelines and good for referrals. The picture may be different in other countries that have private healthcare - I don't know.

Peer reviewed literature is full of awesome stuff and full of crap. I read bad papers every day and there are many many bad journals out there. My favourite is the computer scientists who build a random paper generator incorporating lots of jargon that doesn't mean anything, and they got a couple of articles published in peer reviewed literature. Gives you an idea of standards of these particular journals. Medical journals are probably even more pressured by drug companies and big pharma than GPs.

Oh and what REALLY annoys me is bad medical research. It's not the end of the world to have bad research in my field (marine genetics) because it's not going to KILL anyone. Bad medical research does, however.

netchik

Great post, although I'd have to disagree with the GP bit. Not in the UK. They are allocated 7 minutes to have you in and out... how can a "factory worker" (albeit with a degree) understand your issues well enough to make a sensible diagnosis? That's why forums like this are great. Discussion, backed up by a LOT of research and getting to know your own body have to be the best starting point. That way, when you go to the GP, you can badger them for the tests you want/advice you need etc.

missy_1975

Good post and very good advice.

I'd like to emphasise the point on research papers. I've seen quite a lot of people on the forums refer to the abstract of a research paper and proffer that up as advice. But there are even more people who listen to that first guys posting and then go off quoting HIM as an authority. Madness.

Good example this week is a reference to a paper where the abstract concludes that Low GI or High GI diets are no different for weight loss. If you take an hour to read the paper you realise that the test was in no way relevant to the fat loss programmes people are undertaking in the real world - very briefly it was measuring the fat loss 9 months after cessation of dieting, oh and the diet itself was in the 100-300 caloric defecit range. Amusingly it dismissed the fact that the low GI group lost 50% more weight as statistically irrelevant.

Which brings me on to my second point. Whilst I agree with you about trusted sources, something you have to bear in mind is that those sources actually take a very long time to adopt new practices. With good reason in many cases but it's still typically well over 10 years for evidence based medical practice to be adopted.

So I would augment your excellent post with the statement that it's not a bad thing to consider other sources so long as you review it thoroughly and draw your own conclusions.

:drinker:

lulabox

Great post, although I'd have to disagree with the GP bit. Not in the UK. They are allocated 7 minutes to have you in and out... how can a "factory worker" (albeit with a degree) understand your issues well enough to make a sensible diagnosis? That's why forums like this are great. Discussion, backed up by a LOT of research and getting to know your own body have to be the best starting point. That way, when you go to the GP, you can badger them for the tests you want/advice you need etc.

Whenever I've had something serious, I was looked after correctly and referred to specialists. For other stuff, 7 minutes...

Aperture_Science

I love this thread. Thank you so much for posting it. I’m a relative newbie here but I’ve been a member of a similar paid for site and your observations would have been just as valid for that site. This is what I wrote on their discussion forum in the midst of an argument about artificial sweeteners in soft drinks.

“As someone who is trained in science and has a reasonable grasp of reading research papers I (hopefully) am reasonably well equipped at spotting the rubbish. But it’s more difficult for people not involved in science. For those I would offer 5 pieces of advice.
Ignore any article which:
*Eludes to some conspiracy that “they” are trying to cover up.
*Tries to sell you something.
*Does not give clear references to where they got the data from.
*Uses statements like “As everyone knows”, “it is common sense that...”, or “As generations before us have known...” (it’s not that these phrases are bad in themselves it’s just that anyone who wrote a serious, peer reviewed article or paper, and included, say “It is commonly known that eating more calories than you expend will lead to weight gain.” The reviewer would immediately reply with “If it is commonly known please provide a citation to the key experiments and papers which prove this point”. Therefore phrases like those above indicate that the article or paper has not been reviewed or has been reviewed by people who do not have a clue.)
*If it’s on the front page of the Daily Mail or Telegraph laugh it off. <non UK members may not get this little tongue-in -cheek dig at the UK press>

You post is brilliant and I wholeheartedly support your book choices:

Bad Science http://www.amazon.co.uk/Bad-Science-Ben-Goldacre/dp/0007240198http://www.amazon.co.uk/Bad-Science-Ben-Goldacre/dp/0007240198
How to lie with statistics http://www.amazon.co.uk/How-Lie-Statistics-Penguin-Business/dp/0140136290/ref=pd_sim_b_21

lulabox

“As someone who is trained in science and has a reasonable grasp of reading research papers I (hopefully) am reasonably well equipped at spotting the rubbish. But it’s more difficult for people not involved in science. For those I would offer 5 pieces of advice.
Ignore any article which:
*Eludes to some conspiracy that “they” are trying to cover up.
*Tries to sell you something.
*Does not give clear references to where they got the data from.
*Uses statements like “As everyone knows”, “it is common sense that...”, or “As generations before us have known...” (it’s not that these phrases are bad in themselves it’s just that anyone who wrote a serious, peer reviewed article or paper, and included, say “It is commonly known that eating more calories than you expend will lead to weight gain.” The reviewer would immediately reply with “If it is commonly known please provide a citation to the key experiments and papers which prove this point”. Therefore phrases like those above indicate that the article or paper has not been reviewed or has been reviewed by people who do not have a clue.)
*If it’s on the front page of the Daily Mail or Telegraph laugh it off. <non UK members may not get this little tongue-in -cheek dig at the UK press>

Thanks and this is great advice!! Especially the Daily Mail comment hehehe :P

meerkat70

*Eludes to some conspiracy that “they” are trying to cover up.

Alludes surely? Unless they're trying to hide the cover up of the cover up of the conspiracy.

I do hope your reading of scientific journals is more careful than you choice of words. :-)

(Stand back, man. I *am* a scientist....)

engineman312

BioQueen

I 110% agree with this. Scientific literature can easily be misinterpreted or spun to meet the needs of a specific school of thought regarding exercise and nutrition. Even after reading studies from the primary sources they can be difficult to apply to practical purposes. However, they are the only sources with scientific backing.

Aperture_Science

*Eludes to some conspiracy that “they” are trying to cover up.

Alludes surely? Unless they're trying to hide the cover up of the cover up of the conspiracy.

I do hope your reading of scientific journals is more careful than you choice of words. :-)

(Stand back, man. I *am* a scientist....)

Hehehe... you got me!

LiveEnjoyEndure

Seriously doctors! Peer Reviewed Seriously!

I am a scientist trust me ... LOL.

This web site is about people supporting each other to make changes to their lives. IMAO ... you don't need a PhD to use this site, just a bit of common sense, which in my experience people with PhDs tend to lack.

BTW: It is anal sphincter curling the amount of times you have mentioned your PhD in this thread!

Collinsky

Seriously doctors! Peer Reviewed Seriously!

I am a scientist trust me ... LOL.

This web site is about people supporting each other to make changes to their lives. IMAO ... you don't need a PhD to use this site, just a bit of common sense, which in my experience people with PhDs tend to lack.

BTW: It is anal sphincter curling the amount of times you have mentioned your PhD in this thread!

I wish there was an emoticon that stood up and cheered. I'd use it now.

cessnaholly

bump

pixlamarque

Seriously doctors! Peer Reviewed Seriously!

I am a scientist trust me ... LOL.

This web site is about people supporting each other to make changes to their lives. IMAO ... you don't need a PhD to use this site, just a bit of common sense, which in my experience people with PhDs tend to lack.

BTW: It is anal sphincter curling the amount of times you have mentioned your PhD in this thread!

Amen. :flowerforyou:



Personally, I consider it positive population control. Dummy takes bad advice from random internet poster>>dummy has bad result>>less dummies in the world. Score.

pixlamarque

Besides, I see lots of people on here who have medical doctors recommending HCG diet and the like. That really takes a Ph.D. There are many good doctors out there but there are many bad ones as well and most of the ones I know (and I've worked in medicine for nearly 20 years) do not know squat about diet, nutrition, and healthy weight loss.