Using science to argue your case

JessG11

Snark aside, I am a neuroscientist and I am genuinely curious why the general public does not trust science/peer review, and what you trust instead. It makes me sad to see that people basically think that my life's work is worthless.

Also, serious offer, if anyone on here wants help with reading and interpreting any particular research article, please PM me, I'm happy to help. Sometimes if you only read the abstract, that's not helpful. The most important sections are the methods and results.

^^^^^^YES!^^^^ I don't think your life work is worthless. Research is a big part of what I do as well!

I've taken undergrad and graduate level classes where I've been taught to break down studies and articles, and how to understand methods and results. I believe you have to understand the ins and out and designs of studies. It's not easy....or it isn't always for me (though I'm sure it can be for others). But I believe you have to break these studies down to be able to fully understand all the data, along with how and from who/what it was collected, along with the design and methods. I'm not sure why people knock peer reviewed articles, as someone on here posted...I'd prefer peer reviewed articles. To me, it tells me that others in the same area have picked the study apart.

karlahere

Please don't take this personally, but here are my 2 cents.

1. There is more than likely a lot of unpublished or non-peer reviewed scientific articles out there in all disciplines that have very relevant information, because the findings in these studies go against the mainstream thought on that subject, or the results are not what the company contracting the study wants to see.

A lot of these papers did not get published because they're: 1) using questionable methodology; 2) using really bad statistical acrobatics; and 3) not repeatable by any other independent attempts to replicate the experiment. An academic journal will definitely put up all these rigorous barriers because they themselves will catch flak if a study they published turned out to be bad science. The actual tobacco studies in the last half century all came to the conclusion that smoking is a major cause of lung cancer--even those funded by the tobacco companies got the same results. But it was suppressed by the tobacco companies, not by the academia.

Just because something is published through a reliable source doesn't necessarily mean it's 100% true, and just because something isn't published through a reliable source doesn't necessarily mean it's 100% false.

And these are exceptions rather than the norm. The golden standard of science is getting published. Anyone is free to refute these studies, and other journals are happy enough to publish refutations. So there's a check and balance, at least for the hard sciences. And what doesn't get published needs improvement.

2. Scientists have the ability to alter time-lines and control groups in order to achieve certain results, or to make a result appear to be positive when it's not. I'm not saying all scientists do this, and I'm assuming that most don't, but the possibility is still there.

Bias is an inherent human value--it's the only way we can filter out white noise. That's why there are accepted methods of filtering noise, and scientists are obligated to declare any assumptions/controls/factors that may directly or indirectly affect the outcome of the study. Everything must be stated, down to the brand and model of the equipment they used, and the source of funding. That way, their biases are declared in the open, and there's little room for massaging the data.

3. Conflict of interest. Again back to the GMO debate - if a company makes a new product and the same company is the one to test it to see if it's safe, is that not a terrible conflict of interest, and thus not entirely reliable? Third parties/independent studies I feel would be more reliable, but as mentioned before, if the results do not match the agenda, they can and most likely will be discredited.

Third parties are indeed called upon when testing drugs, GMO, etc. In fact, most safety/toxicity/allergen/etc tests for GMOs are conducted by multipartite bodies. GMOs are rigorously tested, unlike traditionally bred cultivars whose mutations are poorly documented and distribution is not regulated ("organic" foods included).

4. Science can't prove everything, or at least not yet. Why do holistic approaches to medicine still work when there is no evidence that they do? Why do some children with autism that have changed to an organic diet have less problems with their autism?

And that's the beauty of science. We keep discovering, soldiering on into the abyss of even more questions just as we've answered one question.

Most lines of evidence are anecdotal for the holistic approach, and there's very little correlation to even dream of claiming causation. Claims like rapeseed oil prevents cancer, or organic veg is has more antioxidants are products of bad, even pseudo-science. These guys do not have controlled (meaning null, or your placebo) subjects to test whether A produces a substantial response to the population receiving the supplement. There's a landslide of papers that refute the claim that organic veg have better benefits than non-organic. And what counts as organic could actually be anything as the labelling legislation is not airtight enough to regulate it.

People want to be more aware, and people are aware that things are being hidden, and that's why they don't always trust science.

What do scientists have to present before these people are satisfied that they're getting the truth? Even after the World Health Organization--a non-partisan, international body--has issued out a statement that GMOs are safe, people still lobby against it.

JessG11

There is a lot of crap in scientific journals too, and peer reviewed ones at that.

Quacks have their own journals with very convincing looking 'studies'.

Most studies in peer reviewed journals can't ever be repeated.

I agree with this along the lines of certain things....like Coke saying aspartame is completely safe because of "their" studies. But, I've been taught that if the study can't be repeated, it's not a good study and should never be used. So to me, saying most peer reviewed studies can't be repeated isn't right. Peer reviewed, (though yes some may be bad) means the study has been picked apart and I myself would think that would be addressed.

shannashannabobana

'

People do not trust diet science because we have gotten years of bad science or bad conclusion pushed on the public that did not work. That doesn't mean all science is bad.

It's not just diet science that people don't trust. From my meanderings across the Internet, it's basically any science... though some of the big "untrusted" fields are diet, climate, and health (not diet related).

Probably because those are the ones that have had huge reversals or have tried very hard to affect public policy in ways people don't like. If you make a prediction and it doesn't come true...you need to go back and figure out what you did wrong not just push it harder.

chezjuan

One non-diet example of this for me is Dark Matter and Dark Energy. To me, it sounds like a cop-out... "Hmmm, the mass umbers are wrong on a galactic scale, but if we change them by adding in 'dark matter' that does nothing but add mass, then the numbers work! Therefore dark matter! Now we just have to find some."* However, it is the general consensus, and nobody has come up with a better explanation, so I will accept it for now. Were I an astrophysicist, I would probably be trying to figure out if there is an alternate explanation that makes more sense to me (but I may also have come to understand the equations and why dark matter is the best option).

*I admit that is a way-over-simplified view of the whole dark matter argument.

Haha, I understand how the terms sound somewhat fishy, but they're pretty descriptive. Without getting into many details, I'll say a few words, just because I like this area so much. There is evidence from multiple sources that the mass in the universe is more than the mass we "see" through telescopes etc, i.e. there is mass that only feels gravity and no other forces, but it's not clear what exactly it is. There are many (way too many) possible candidates and theories but the data are not conclusive up to now. So it's "dark matter" (i.e. non-radiative matter) until this settles. As for dark energy, it's more difficult to explain, but let's say the way the universe expands tells us there is some "background" energy that is neither mass nor radiation. And it's actually most of the universe's energy, about 70%. The thing is, it's very difficult and expensive to do experiments, and the theories we have up to now are incomplete and break down for the very early universe. There are a lot of possible explanations, way too many, but the problem is we cannot rule out most of them.

Thanks! That is probably the best explanation I have seen. Most articles I see either go way too technical for my non-physics brain or just gloss over the details.

Beastette

Just go with Broscientology. None of those pesky words, no multisyllabic utterances, no higher mathifications.

Because broscience.

ryry_

Also, if people could just stop saying "Because science" as as answer that would be great

Eat a calorie deficit to lose weight...Because Science

Snark aside, I am a neuroscientist and I am genuinely curious why the general public does not trust science/peer review, and what you trust instead. It makes me sad to see that people basically think that my life's work is worthless.

Also, serious offer, if anyone on here wants help with reading and interpreting any particular research article, please PM me, I'm happy to help. Sometimes if you only read the abstract, that's not helpful. The most important sections are the methods and results.

When in comes to this arena, people will always trust their confirmation bias

JessG11

I'd like to point out something that is definitely true for my area of expertise (physics) but perhaps in bio-medical sciences that's different, so please correct me if that's not the case. The frontline of research deals with things that are not well understood by scientists and there are conflicting theories and experiments and an ongoing scientific discussion about what's going on. These papers, even though they are important and sometimes ground breaking for the scientific world, are basically useless for general public. Well, the public has no interest for physics research, but high-tech industry is interested for sure. For example, a large part of today's research (in solid state physics) is related to / motivated by the physical realization of quantum computers. There's a huge number of papers published everyday, but there is really no point for people in industry following all these. Once things get clarified a bit, once there are some promising candidates then, yes, for sure. But until then the discussion stays within the scientific circles and for good reason: things are so unclear, there's no practical use of the work done up to now. I'd expect the same would be more or less true for other disciplines. The frontline of research is interesting because of the progress being made, but for practical uses, one needs to go one step back to knowledge that is well founded. If there is an ongoing discussion about an issue, you can't really pick one paper and draw conclusions out of it. You need to understand and follow the work done by the whole community and even then, that doesn't mean you can reach any conclusion.

(Edited for typos)

This is a great post....though I am not in the physics world and probably couldn't begin to understand what you do, it's the same in my field. It's my ethical duty to keep up with what is being done, written, studied in my field. I want to know what the best practices are. If it wasn't for studies, we wouldn't have advancements in my field. LOVE your last sentence!

IronSmasher

When it starts to get too much for me, I go here:

http://theflatearthsociety.org/cms/

IronSmasher

https://www.youtube.com/watch?v=C88y6q0zylg

plantboy2

DRINK LIQUID CHLOROPHYLL!

My friend says it works.....

Great post OP

nomeejerome

nice read

JessG11

Snark aside, I am a neuroscientist and I am genuinely curious why the general public does not trust science/peer review, and what you trust instead. It makes me sad to see that people basically think that my life's work is worthless.

Also, serious offer, if anyone on here wants help with reading and interpreting any particular research article, please PM me, I'm happy to help. Sometimes if you only read the abstract, that's not helpful. The most important sections are the methods and results.

I'm not going to speak for your field (because I've never read any peer-reviewed journals there or understand the publication process) but in many fields in the social sciences, for example, only articles that find a statistically significant effect (with the arbitrary p<.05) get published. So, authors are inclined to search for a hypothesis that will get them published, irrespective of any mounting evidence that refutes the current relationship they observe in their own study. The magnitude or size of these effects can also be overstated depending on model specification used to analyze the data, the observations that are removed from the study, etc. A lot of the time what we see is someone make an incredulous finding that gets them published, and then you will find a prompt backlash from the scientific community demonstrating that the effect shown is highly dependent on the factors specified- and we return to the status quo. Unfortunately, what makes headlines on TV are these “amazing” articles that get refuted later on, but that never makes it to the media…I think that might be why the public is leery of statistics or “science”- I think it’s more of a reflection of the media’s lack of follow-up and sensationalizing particular findings over others.

I am in the social science field (completing my LCSW) and I don't like all of this statement but I think I understand what you are saying. There is a lot of "pop psychology" out there and some of it actually can make sense. But in the social science field, we are "supposed" to be taught how to see around this. We are taught to use research to find the best practices and use those. We don't get best practices from extreme claims that get the media popping....though personally I find some of it interesting, I don't take it as fact or use it in my own personal research.

IronSmasher

DRINK LIQUID CHLOROPHYLL!

My friend says it works.....

Great post OP

I shall use my body (it's my best advicer) to see if that's right for me! Thanks for the tip!

ninerbuff

Every day the MFP boards bring us a selection of threads from people claiming this food group is bad/that food group is wonderful. Every thread is then filled with a selection of arguments for and against the OPs initial supposition. Many will claim a variety of evidence or anecdotes in order to support their stance, though rarely will that evidence be produced or linked. On the odd occasion there is a link but the source is often a YouTube video or a blog post. This irritates me and I think you all can do better.

A quick search on Google or Google Scholar can bring you all sorts of information on how to critique medical scientific claims and research and how to use it to employ best practice. A lot of this is a bit wordy so here are some highlights from a Wikipedia article as it’s very good for simplifying language – although I wouldn’t necessarily use it to support a scientific argument as you will see (Tip:- I’ve referenced this article at the bottom so others can read it)

1. “Press releases, blogs, newsletters, advocacy and self-help publications, and other sources contain a wide range of biomedical information ranging from factual to fraudulent, with a high percentage being of low quality”

2. “The popular press is generally not a reliable source for scientific and medical information in articles. Most medical news articles fail to discuss important issues such as evidence quality,[11] costs, and risks versus benefits,[12] and news articles too often convey wrong or misleading information about health care.[13] Articles in newspapers and popular magazines generally lack the context to judge experimental results. They tend to overemphasize the certainty of any result, for instance, presenting a new and experimental treatment as "the cure" for a disease or an every-day substance as "the cause" of a disease. Newspapers and magazines may also publish articles about scientific results before those results have been published in a peer reviewed journal or reproduced by other experimenters. Such articles may be based uncritically on a press release, which can be a biased source even when issued by an academic medical center.[14] News articles also tend neither to report adequately on the scientific methodology and the experimental error, nor to express risk in meaningful terms. For Wikipedia's purposes, articles in the popular press are generally considered independent, primary sources.”

3. In-vitro studies and animal models serve a central role in biomedical research, and are invaluable in elucidating mechanistic pathways and generating hypotheses. However, in vitro and animal-model findings do not translate consistently into clinical effects in human beings. Where in vitro and animal-model data are cited on Wikipedia, it should be clear to the reader that the data are pre-clinical, and the article text should avoid stating or implying that the reported findings necessarily hold true in humans. The level of support for a hypothesis should be evident to the reader.
Use of small-scale, single studies make for weak evidence, and allow for easy cherry picking of data.

4. Scientific journals are the best place to find primary source articles about experiments, including medical studies. Every rigorous scientific journal is peer reviewed. Be careful of material published in a journal that lacks peer review or that reports material in a different field. (See: Martin Rimm.) Be careful of material published in disreputable journals or disreputable fields

http://en.wikipedia.org/wiki/Wikipedia:Identifying_reliable_sources_(medicine)

Ok so once you have found your peer reviewed study published in a respected scientific journal, how can you tell that the study gives you enough evidence to argue your case on an MFP thread? Medical scientific evidence is graded from 1 to 5, with level 1 evidence being the most robust. That’s not to say that level 1 evidence dating back 20 years is still relevant as science is constantly changing based on whats observed from current data. Any how what are these levels of evidence then? I’ve copied a table from the National Health and Medical Research Council from Australia, but it is standard terminology throughout the world.

In simple terms, one way of looking at levels of evidence is as follows (the higher the level, the better the quality; the lower, the greater the bias):
I. Strong evidence from at least one systematic review of multiple well-designed randomised controlled trials.
II. Strong evidence from at least one properly designed randomised controlled trial of appropriate size.
III. Evidence from well-designed trials such as pseudo-randomised or non-randomised trials, cohort studies, time series or matched case-controlled studies.
IV. Evidence from well-designed non-experimental studies from more than one centre or research group or from case reports.
V. Opinions of respected authorities, based on clinical evidence, descriptive studies or reports of expert committees.

There is a lot more too it than that of course, but all we are trying to do is to win an argument on a social media page about diet, so that should be enough to be going on with.

One final tip – avoid use of logical fallacies such as straw man arguments, ad-hominem, appeals to authority etc. Not only will you be shot down but you will look like a pillock.

Love this. Of course there will always be people who will never concede to science due to anecdotal evidence from their own experiences that aren't journaled.

A.C.E. Certified Personal and Group Fitness Trainer
IDEA Fitness member
Kickboxing Certified Instructor
Been in fitness industry for 30 years and have studied kinesiology and nutrition

ninerbuff

There is a lot of crap in scientific journals too, and peer reviewed ones at that.

Quacks have their own journals with very convincing looking 'studies'.

Most studies in peer reviewed journals can't ever be repeated.

Which is why one shouldn't just rely on one study, but several to make a more logical decision.

A.C.E. Certified Personal and Group Fitness Trainer
IDEA Fitness member
Kickboxing Certified Instructor
Been in fitness industry for 30 years and have studied kinesiology and nutrition

sharonfoustmills

as a doctoral student, I've learned well what is and is not considered reliable sources, but this is something most people who do not move past a bachelors degree (unless the bachelors is in a research based science) are completely unaware of

I dare say you won't change these people with this information though. They do not care to know the accuracy of their information- it is more important to them to get to make the claim that they know.

ninerbuff

Listen to your own body, it's your best advicer and decide for yourself what's best for you!

Unfortunately, lots of people with eating disorders follow this advice.

A.C.E. Certified Personal and Group Fitness Trainer
IDEA Fitness member
Kickboxing Certified Instructor
Been in fitness industry for 30 years and have studied kinesiology and nutrition

toutmonpossible

There also are articles that explain the difference between different studies, i.e., treatment studies v. observational studies. http://en.wikipedia.org/wiki/Clinical_study_design

These are all good guidelines to have. But a person's individual experience still can be valuable. Studies are expensive. Studies usually are paid for by drug or food corporations that will benefit. Studies tend to focus on the most serious issues of life and health or physical performance. Studies notoriously have had only male subjects, often young male athletes or middle-aged unhealthy men and results for women were expected to be extrapolated from them. It is perfectly possible to have insights for which there will be no study.

So it can never be just a battle of who can cite the better study.

ninerbuff

Please don't take this personally, but here are my 2 cents.

1. There is more than likely a lot of unpublished or non-peer reviewed scientific articles out there in all disciplines that have very relevant information, because the findings in these studies go against the mainstream thought on that subject, or the results are not what the company contracting the study wants to see.

A lot of these papers did not get published because they're: 1) using questionable methodology; 2) using really bad statistical acrobatics; and 3) not repeatable by any other independent attempts to replicate the experiment. An academic journal will definitely put up all these rigorous barriers because they themselves will catch flak if a study they published turned out to be bad science. The actual tobacco studies in the last half century all came to the conclusion that smoking is a major cause of lung cancer--even those funded by the tobacco companies got the same results. But it was suppressed by the tobacco companies, not by the academia.

Just because something is published through a reliable source doesn't necessarily mean it's 100% true, and just because something isn't published through a reliable source doesn't necessarily mean it's 100% false.

And these are exceptions rather than the norm. The golden standard of science is getting published. Anyone is free to refute these studies, and other journals are happy enough to publish refutations. So there's a check and balance, at least for the hard sciences. And what doesn't get published needs improvement.

2. Scientists have the ability to alter time-lines and control groups in order to achieve certain results, or to make a result appear to be positive when it's not. I'm not saying all scientists do this, and I'm assuming that most don't, but the possibility is still there.

Bias is an inherent human value--it's the only way we can filter out white noise. That's why there are accepted methods of filtering noise, and scientists are obligated to declare any assumptions/controls/factors that may directly or indirectly affect the outcome of the study. Everything must be stated, down to the brand and model of the equipment they used, and the source of funding. That way, their biases are declared in the open, and there's little room for massaging the data.

3. Conflict of interest. Again back to the GMO debate - if a company makes a new product and the same company is the one to test it to see if it's safe, is that not a terrible conflict of interest, and thus not entirely reliable? Third parties/independent studies I feel would be more reliable, but as mentioned before, if the results do not match the agenda, they can and most likely will be discredited.

Third parties are indeed called upon when testing drugs, GMO, etc. In fact, most safety/toxicity/allergen/etc tests for GMOs are conducted by multipartite bodies. GMOs are rigorously tested, unlike traditionally bred cultivars whose mutations are poorly documented and distribution is not regulated ("organic" foods included).

4. Science can't prove everything, or at least not yet. Why do holistic approaches to medicine still work when there is no evidence that they do? Why do some children with autism that have changed to an organic diet have less problems with their autism?

And that's the beauty of science. We keep discovering, soldiering on into the abyss of even more questions just as we've answered one question.

Most lines of evidence are anecdotal for the holistic approach, and there's very little correlation to even dream of claiming causation. Claims like rapeseed oil prevents cancer, or organic veg is has more antioxidants are products of bad, even pseudo-science. These guys do not have controlled (meaning null, or your placebo) subjects to test whether A produces a substantial response to the population receiving the supplement. There's a landslide of papers that refute the claim that organic veg have better benefits than non-organic. And what counts as organic could actually be anything as the labelling legislation is not airtight enough to regulate it.

People want to be more aware, and people are aware that things are being hidden, and that's why they don't always trust science.

What do scientists have to present before these people are satisfied that they're getting the truth? Even after the World Health Organization--a non-partisan, international body--has issued out a statement that GMOs are safe, people still lobby against it.

:drinker:

A.C.E. Certified Personal and Group Fitness Trainer
IDEA Fitness member
Kickboxing Certified Instructor
Been in fitness industry for 30 years and have studied kinesiology and nutrition