Does anyone on MFP actually understand studies?

pattyproulx

I'm a numbers guy and I love stats so to me this stuff is interesting.

The difficult thing is that it seems scientists and researchers don't seem to understand this themselves (or at least avoid using their understanding of it so that they can come to certain pre-defined conclusions).

It's absolutely everywhere. Not to open a can of worms, but as an example, look at all the pro-whole-grain studies.
The way they generally deem whole-grains to be healthy is by looking at whole-grain eaters vs. non-whole-grain-eaters and assessing their health - completely disregarding all the other lifestyle factors involved.

Also, what is released to the masses is usually a pretty small excerpt of the actual study and even the conclusions found are increasingly inconsistent with the information in those studies. The media also tends to further skew the results of studies to make them more interesting. Anyways, the whole thing is a mess.

The fact remains that in the real world being able to conduct thorough studies is very time consuming and very expensive so most of these studies need to be funded by corporations who have a stake in the outcome (and unfortunately, that almost always biases the results).

BinaryPulsar

Yeah, as a person with a doctorate in toxicology, I'd say I have a pretty good grasp on what a study says and how to interpret it.

That's not to say I've read everything about anything in my field.

And as the first reply says it is vital who funds it, as you design a lot of experiments to make them say what you want.

Lol as someone earning a doctorate in synthetic chemistry, I find it a lot harder to design my experiments to say what I want. You can fabricate data to do that, but generally we frown upon such behavior in my field.

As for the topic of this post, the process which information passes from scientist to general public leads inevitably to this type of behavior. Scientist publishes XYZ with ABC restrictions and DEF alternatives to be explored. Science news says XYZ with ABC restictions, popular news publishes "SCIENTISTS CLAIM ABC!!" and the general public thinks "OMG I bet ABC can cause TUV!"

My husband is a physicist, so obviously a very different area of science. I agree with this. There is no fabrication. They are trying to make things work and happen, so there would be no reason to not be honest...that would hurt their own experiments, lose the respect of other scientists, and get them to lose their funding that they spent years working towards. And of course it is part of his job to be regularly published in peer reviewed journals. But, whenever the news reports on them it often involves them saying things that are incorrect (NPR is better about that). The newscaster and/or journalist does not understand the physics or know what they are reporting on. Then the public responds acting like they know everything about it and they don't at all (all kinds of wacky misinformation). We just have to accept it and don't worry about.

taylor5877

Really the only dishonest thing we ever did in our lab was not publish the stuff that didn't work.

That's not really our fault though. I'm not saying we went out to make up results, but we'd often take things that we knew would work and do a lengthy/thorough experiment to be able to either give a funding coorporation the data they needed/wanted or to get something published.

It doesn't mean we did things unethical to make something we knew untrue true.

I should have clarified.

Like for instance I studied arsenic adsorption on iron oxides. A company gave us a product they made for this purpose and told us to test it. After first identifing what it was (we didn't trust them at their word obviously) we knew it was the same product as one we'd synthesized in our lab to sorb As.

Pretty easy to design some experiments comparing the two ("pure" synthesized product and industrial product) to make the industrial product look "good".

DrMAvDPhD

I'm getting close to the end of my chemistry and physics degrees now, and I'm still working in the local supermarket. I've come to the conclusion that the average person doesn't know what correlation and causality mean, No point asking if they know the difference between them.

Plenty of these studies that appear around the place are perfectly valid. The problem is (as per the original post) that the average person doesn't know how to read/interpret them correctly.

my favorite:

DrMAvDPhD

Really the only dishonest thing we ever did in our lab was not publish the stuff that didn't work.

That's not really our fault though. I'm not saying we went out to make up results, but we'd often take things that we knew would work and do a lengthy/thorough experiment to be able to either give a funding coorporation the data they needed/wanted or to get something published.

It doesn't mean we did things unethical to make something we knew untrue true.

I should have clarified.

I didn't mean to imply that you were unethical. Just that it is more difficult for some branches of science to do such things than others. I honestly think there should be a journal of "Really cool ideas that didn't pan out" so that I could publish more often!

As for publishing things for funding agencies, I have only used funds from NSF and NIH and while they expect results, I have never personally felt pressured to get particular results. Perhaps my advisor shields me from that.

kiminikimkim

The most interesting study I found was on Vitamins.

Those who took vitamins lived shorter lives than those who don't. Had the study been more positive, I am sure all the vitamin suppliers would have used this in their advertising.

Icelandic_Saga

The most interesting study I found was on Vitamins.

Those who took vitamins lived shorter lives than those who don't. Had the study been more positive, I am sure all the vitamin suppliers would have used this in their advertising.

I'll be darned....Glad I don't take Vitamins! lolol

Long_and_Lean

Fun fact: I am at this very moment enduring a mandatory lecture on biomedical research integrity.

taylor5877

I so would have enjoyed my seminars more if smart phones would have been more widespread...

Of course, you could just be surfing from a laptop and making it look like you're working.

I played a few games of chess during seminars that way....when I was lucky enough to get a seat in the back.

pattyproulx

The most interesting study I found was on Vitamins.

Those who took vitamins lived shorter lives than those who don't. Had the study been more positive, I am sure all the vitamin suppliers would have used this in their advertising.

I'll be darned....Glad I don't take Vitamins! lolol

Gah, don't mean to use you as an example, but this is exactly what I meant in my previous post. You can`t come to any conclusions from those types of studies.
You can`t just look at a random set of people, isolate one variable, group people up and come to conclusions but this is the type of stuff that happend all the time! Then the media gets a hold and makes a big story out of it.

For this example, it can probably be explained pretty easily. Doctors will often tell their sick patients that they really should start taking more vitamins so that ups the sick population taking vitamins. Then you will have people who eat crap and don`t eat many fruits or veggies and take vitamins in the hope/belief that they're being healthy. People who eat a ton of veggies and fruit probably don't take many vitamins because they get the nutrients they need in the food they eat. So you have sick people and people who eat crap that die earlier than people who get their vitamins from fruits and veggies.

These are all possible explanations for the result. You can't come to any conclusions about vitamins based on this. This isn't to say vitamins are good. They could be bad for you for some unkown reason, but either way you can't tell anything based on this study. These are the types of studies we need fewer of (though it can 'hint' at a subject for a future study).