Welcome to Debate Club! Please be aware that this is a space for respectful debate, and that your ideas will be challenged here. Please remember to critique the argument, not the author.
Do you read with a sense of skepticism?
Replies
-
What's humorous about statements such as these is that there is no logical justification.
No X does Y, so why does Z do Y?
Applying this same theoretical logic with any other variables usually exposes woo with ease.
There is no animal on the planet that practices medicine.
There is no animal that travels using mechanical means.
There is no animal on the planet that deliberately sacrifices it's present for it's future.
By this logic, stop using the internet as no other animal runs an ISP.
No other animals use tools or practice farming to the extent that humans do. Seems like consuming dairy is a part of evolution being there are many cultures who still have intolerance/almost zero capacity to handle dairy (mainly Asians) whilst others have adapted some form of tolerance (quantity of consumption matters/limited amount of lactase production). Sure, the GI tract pH change from infant to adult is still present but this seems like a better adaptation to an omnivore lifestyle/ability to consume a greater variety of food for survival & immunity2 -
Yes, but one issue is that the way the media tries to make everything new and interesting (or just clickbaity) means that many people are confused about this and think everything is constantly changing.
I can think of some changes, but the main advice has been pretty consistent.
The root of the problem is lack of science education. Unless you actually have a degree in the sciences (I have a degree in molecular biology and have worked in research) you don't really learn in school how to parse scientific research, let alone reading actual papers. Even if people go to college in most cases the non-major science classes are subpar. Clearly there is a major failure in the education system at teaching people what is a reliable source and what isn't. The job of the media is to get views, and while many of them do maintain some semblance of journalistic integrity while also being kind of clickbaity, a lot of them don't. Printing the same common sense dietary recommendations isn't going to get views. Printing stuff about how scientists are a bunch of money-hungry industry shills, global GMO and fluoride conspiracies, and everything both causing and curing cancer is going to get views. When we don't have adequate science education and we have the media sensationalizing anything, is it any wonder so many people distrust research and just ignore whatever research doesn't support what they want to hear?1 -
I'm not trained as a medical doctor. I'm not trained as a dietician. I don't even have a certificate of certification in certified nutritionism. What I do have is an ability to read. I look for information such as citations, sources, and the credentials of the person doing the writing. For instance. Drs. Axe and Fung do deserve some credibility within the scope of their expertise as long as they stay there, while Dr. Oz has entirely exited his expertise and has no credibility. An ethical expert explains the limits of his expertise. This is not all. There are autodidacts who have accumulated credibility by study and rigorous experimentation. Beyond that, it is important to me that I question the interest of the presenter of the information.3
-
1 -
I've seen March of the Penguins, and those birds don't use fire to cook their food. Neither do my cats. I've heard stories about bears in Yosemite breaking into people's cars to steal food, but I've never heard of a bear starting a camp fire and hearing s'mores. (I think Smokey would disapprove.)
But nobody has ever told me we should stop cooking.2 -
Always with skepticism! If I read an article saying dairy is bad I can find another article saying how amazing dairy is. I'm a vet, I have a medical background and some knowledge regarding evidence based medicine and how to assess evidence.
I asked my own GP about his thoughts on a dairy or gluten free diet. His response: if you feel better eating that way then great, that's suitable for you. If you don't feel any different then you are clearly fine to continue eating dairy and gluten.
And good point about cats drinking milk (of another species!). I've seen dogs eat sheep placenta- does that mean they are going against the natural order of things??
Interesting topic!0 -
No not at all. I like to become one with what I read and then fall asleep and let my dreams and nightmares do the filtering.
Last night I dreamed I was back in the Air Force, but it was now the Space Force, and I was up in space and there was a problem with the air, and we were struggling to get back to Earth.
I had fallen asleep to Orson Scott Card's "Wyrms." I suspend disbelief for that sort of reading
2 -
laurenq1991 wrote: »Yes, but one issue is that the way the media tries to make everything new and interesting (or just clickbaity) means that many people are confused about this and think everything is constantly changing.
I can think of some changes, but the main advice has been pretty consistent.
The root of the problem is lack of science education. Unless you actually have a degree in the sciences (I have a degree in molecular biology and have worked in research) you don't really learn in school how to parse scientific research, let alone reading actual papers. Even if people go to college in most cases the non-major science classes are subpar. Clearly there is a major failure in the education system at teaching people what is a reliable source and what isn't. The job of the media is to get views, and while many of them do maintain some semblance of journalistic integrity while also being kind of clickbaity, a lot of them don't. Printing the same common sense dietary recommendations isn't going to get views. Printing stuff about how scientists are a bunch of money-hungry industry shills, global GMO and fluoride conspiracies, and everything both causing and curing cancer is going to get views. When we don't have adequate science education and we have the media sensationalizing anything, is it any wonder so many people distrust research and just ignore whatever research doesn't support what they want to hear?
It's not a problem unique to science media, though. I see it in lots of areas I know stuff about.
I think teaching about vetting sources, how to research, so on, in a broad way would be helpful. (I agree that more and better science education would be good too, but I don't think we can expect the average person to be able to carefully read a scientific study -- in part because they won't put in the time or will prefer what they want to hear.)2 -
NorthCascades wrote: »I've seen March of the Penguins, and those birds don't use fire to cook their food. Neither do my cats. I've heard stories about bears in Yosemite breaking into people's cars to steal food, but I've never heard of a bear starting a camp fire and hearing s'mores. (I think Smokey would disapprove.)
But nobody has ever told me we should stop cooking.
Oh, they are out there. The raw diet people.5 -
It's not a problem unique to science media, though. I see it in lots of areas I know stuff about.
I think teaching about vetting sources, how to research, so on, in a broad way would be helpful. (I agree that more and better science education would be good too, but I don't think we can expect the average person to be able to carefully read a scientific study -- in part because they won't put in the time or will prefer what they want to hear.)
It's definitely not unique to science but science is an especially powerful example because a lot of the other topics out there have more of a degree of subjectivity, whereas with science it's all evidence-based and yet you see so many people out there who have no idea what that actually means. If the evidence doesn't support their argument then they get into global conspiracy territory (from what I've heard even the NutritionFacts.org guy does this, and he actually had a good education, but I guess he knows what the audience will respond to). I agree that the average person can't read a scientific study (the average person probably doesn't even know what the NIH is) but, I think with proper education people could probably be able to read the abstract of a meta-analysis. Of course whether people will actually choose to do that or just remain firm in their beliefs is another question.NorthCascades wrote: »But nobody has ever told me we should stop cooking.
Then you haven't come across the infamous raw vegans of Youtube, or even worse, the raw carnivores and raw milk adherents.
2 -
Just in time: https://www.facebook.com/CollegeHumor/videos/10155690434617807/
Note, may not be safe for work as the f-bomb is dropped.0 -
There is a kernal of truth to the milk thing.
Milk producing mammals produce the enzyme lactase which is required to digest the milk sugar lactose. Lactase production is down regulated with weaning making adolescent mammals lactose intolerant. Humans are a rare exception to this as at some point during our evolution we acquired a mutation in the lactase regulator that meant it continued to be produced into adolescence and adulthood. This mutation ended up being the dominant allele perhaps due to benefits of milk consumption in an agrarian society. Some humans still have the original unmutated pathway and they are lactose intolerant as a result.
Most adult mammal species are lactose intolerant and would not drink milk.
That said so what....says nothing at all about weather drinking milk is good or bad.6 -
NorthCascades wrote: »...When you read things, especially health and diet related, do you have a healthy sense of skepticism?
Oh heck yeah. Although I tend to have skepticism of claims, but also certain aspects of research as well. I am one of those that checks for conflicts of interest when a study is completed, for the agenda of whoever is reporting the claims, of the potential for bias in studies.
And while big established groups do often tend to have more well done research and I might often view them with more trust, they can also be more influenced by those with wealth and power, with 'the need to publish,' because the bigger groups tend to do studies that involve more potential gains in wealth by certain parties. There can also be more pressure on scientists to have findings that are the results their funding entity wants. And that same pressure can extend to those in the scientific publishing industry
I really respect scientists, and what they do. But a number of famous scientists or people in the know have spoken up in the past few years about how broken the science publishing system is currently, to the point that it has put into question many studies, which is REALLY a shame, as there is some really important stuff we're discovering, and without trust in the system, then people feel justified in just saying 'that's fake news,' to whatever they don't like. *
* a couple of the complaints/problems about the publishing system.
- Article from 'The science council and institute of health' - https://www.acsh.org/news/2015/05/19/science-publication-is-hopelessly-compromised-say-journal-editors
- article in scientist.com - https://www.the-scientist.com/uncategorized/is-peer-review-broken-47872
0 -
That said, there is one thing that I run into , a LOT, where skepticism meets, hmmm, I'm gonna say a lack of understanding about the reality of human knowledge of health and medicine.
Basically: there is a difference between 'currently proven to be impossible' and 'impossible as far as we know but new facts are putting that into question,' and 'never been studied.'
There are some health claims made that fly in the face of all knowledge we have, and have no realistic facts to back up any new claims - I view those claims as 'previously known information says this is BS.'
But there are health claims that have evidence to back them up, even when current knowledge would suggest these claims are wrong. If the evidence proves to be valid, then at that point, being skeptical, IMHO, means being skeptical about our CURRENT medical knowledge. We need to look at our current hypothesis about health and re-examine it, to see if we missed anything, and re-examine the new evidence, to see if the new evidence has some flaws.
This kind of thing? I know a lot of people who will treat ANY information that contradicts previously well known science as 'woo,' even when a true scientist would be looking at both old and new information, as it's always a hypothesis, not an 'unchanging fact.'
Admittedly, this one's personal. I have an auto-immune disorder that had symptoms patients reported for decades, that researchers insisted were impossible without even bothering to really ask questions of their patients to explore the subject, essentially. When it WAS finally studied, it showed that patients were absolutely correct; researchers' understanding of the scope of the disorder had been flawed. The researchers got trapped in that mind set of 'we now know everything about this subject, so you must be wrong,' and that's arrogance rather than skepticism, you know?
And that last category, 'never been studied?' I think people don't realize how big a body of information that is, sometimes. Again, this one is personal. I have a rare disorder. It has REALLY not been studied. The majority of symptoms aren't yet listed for it because they have yet to be fully researched, so researchers just don't know. People with the disease will share information about treatments a doctor tried, just to see if it worked, so those can then share it with THEIR doctor, because there's so little research on treatments that the patients are telling doctors what helped as often as doctors are giving suggestions.
Heck, we were excited because just last year, insurance companies finally recognized the disease and assigned it a code so doctors could actually BILL for it properly.
But with something like this...it makes it really, really important to be skeptical about claims that X helped with Y symptom, but you also just have to look at what limited knowledge there is, and make an educated guess about the potential of the claims. There will be no studies verifying it for you. There will be nothing to confirm or deny it works but your own experience and what other lay people like you say.
While this is more extreme that many illnesses, there are a surprising number of disorders and diseases where this still applies: a lot has not been researched and we don't know all of what helps or what hurts.
I think that's hard to understand in today's world, where we have SO MUCH information around that it feels like we know the basics are almost everything. so there MUST be studies to back up claims or they have to be false. And that's...simply not true.
I mean, sure, it could be the case - there's a LOT of BS out there, and many of it lacks studies to prove it. But some things are simply not yet studied. So a lack of supporting studies needs to be weighed against how much research there has been on a particular study, I think, to truly analyze any claims.2 -
I read with a healthy sense of scepticism... the things that I have no bias to agree with or have a bias to disagree with.
I read, with the same rational mind, but with a lot greater predisposition to credibility and lesser predisposition to question things... the things that coincide with or support my own existing beliefs and attitudes.
Really, I think almost everyone thinks they are rational and sceptical and capable of critical thinking but they mostly notice their healthy scepticism when it helps debunk something there wasn't much chance would fool them in the first place.
My sense of scepticism is just that - a sense. What I think saves me most of the time is:
- trying to be self-aware and call out my own biases as much as I can
- using logical thinking and spotting logical falacies (there are a few very common ones that keep propping up on this forum, we should probably have a bingo card for those alone...)
Eta: What I am trying to say is that I think the sense of scepticism is overrated, not in itself but because people feel it a lot less thatn they think and generally not in the situations where they would most need one, to counter their own bias. Practicing the other components of critical thinking, again and again in life, I think would be more helpful in separating truth from fiction.0 -
Skepticism to me basically means questioning. Is there already evidence available to prove or disprove the idea? Does it fly in the face of common sense? Is it too good to be true? Can you trust the expertise of the source?
Anyone who comes to my door or contacts me by phone or email gets questioned. Who are you? What is the donation for? What are the issues you are campaigning on and where do you stand? Do you have a website or additional literature where I can find out more.
I read health, weight loss, and nutrition information the same way. Always interested in new findings but tempered by questions0 -
I am skeptical about everything I read. There’s so much literature out there about how to live, how to eat, how to be a better person, that I think it’s up to our own judgements to make the call.
The problem with the example with the cat and the milk is that milk contains something that makes the young come back to the mother. It’s addictive. I have a cat that I rescued when she was a kitten, abandoned too early, so now she likes to suck on ears and mews like a kitten. I don’t think milk is necessarily unhealthy, but I think species should stick to their own milk.
Generally, my logic when it comes to what I eat, is if I can’t eat it raw without poisoning myself, or if my body reacts negatively to it (from allergies or something overpowering like garlic), I don’t eat it or I eat very little of it. I also don’t eat too much of something that I can’t physically get a lot of (nuts and seeds) or can’t get naturally (sugar and oil).
I know I’m strict about my diet, but I’ve gotten used to it over the years.0 -
NorthCascades wrote: »...When you read things, especially health and diet related, do you have a healthy sense of skepticism?
Oh heck yeah. Although I tend to have skepticism of claims, but also certain aspects of research as well. I am one of those that checks for conflicts of interest when a study is completed, for the agenda of whoever is reporting the claims, of the potential for bias in studies.
And while big established groups do often tend to have more well done research and I might often view them with more trust, they can also be more influenced by those with wealth and power, with 'the need to publish,' because the bigger groups tend to do studies that involve more potential gains in wealth by certain parties. There can also be more pressure on scientists to have findings that are the results their funding entity wants. And that same pressure can extend to those in the scientific publishing industry
I really respect scientists, and what they do. But a number of famous scientists or people in the know have spoken up in the past few years about how broken the science publishing system is currently, to the point that it has put into question many studies, which is REALLY a shame, as there is some really important stuff we're discovering, and without trust in the system, then people feel justified in just saying 'that's fake news,' to whatever they don't like. *
* a couple of the complaints/problems about the publishing system.
- Article from 'The science council and institute of health' - https://www.acsh.org/news/2015/05/19/science-publication-is-hopelessly-compromised-say-journal-editors
- article in scientist.com - https://www.the-scientist.com/uncategorized/is-peer-review-broken-47872
I am a scientist and have been on both sides of peer review...both having my work reviewed for publication and reviewing others work for publication. It certainly is not a perfect system and there is certainly room for bias and laziness amongst reviewers and room for shoddy work to slip by.
I think the public assumes peer review is a more rigorous process than it actually is. I get the sense that people think peer review means that the experts in a field attempt to reproduce the work or something and that certainly isn't the case. Frankly there is way way to much work put up for publication than there are people who have the time to do a real in depth critical review.
Most times what happens is a paper is submitted to a journal, that journal has an editor who is familiar with the field that the journal specializes in and sends that manuscript out to usually three reviewers. These reviewers dont work for the journal, they are just other scientists who work in that field. Given that fields are so specialized often a manuscript can end up in the hands of a reviewer that is only moderately familiar with the field. These reviewers have full time jobs unrelated to reviewing and they dont get any extra time or pay for the act of reviewing. So really the best you can expect is for them to read the entire manuscript armed with the knowledge of the field they have which might not be all that detailed and then just provide critical commentary. That goes back to the editor who decides whether to reject the paper, accept the paper or go back to the authors and ask them to address the reviewers criticisms which can range from not liking a phrasing of a sentence to requesting that an entire new experiment be performed.
All the reviewers have to go off of is what the authors say they did so there has to be some level of trust that the authors are correctly representing their work. Also any one person, even an expert in their field, is unlikely to know everything and can certainly miss dire problems with the work. For example a group might have a statistician who could spot an overly small sample size a mile away but the person from that group reviewing the paper isnt the statistician it's a biologist who can interpret the method used but may not recognize an issue with the sample size. Or maybe the statistician is the reviewer and can catch the issue with the sample size but doesn't catch that the buological method that was used has serious problems.
It is to much to ask of any given reviewer to put in a ton of time to study up on all the specifics and history of the field that the paper they are reviewing relates to. I mean typically speaking a reviewer probably is going to spend maybe half a day on it... not weeks.
Important to realize how many papers are published daily and how many many more are submitted for publication. There is no reward for doing a good job as a paper reviewer and no real punishment for doing a bad job. There is, however, a lot of pressure to finish the review to get back to your actual job. Not sure I see a way around this.
The best analogy I can think of for the peer review system outside of science is jury duty. Science publications are peer reviewed and defendants have a jury of their peers. There aren't professional jurors anymore than there are professional reviewers...they are basically drafted. The quality of "peer" you get and how much time they are going to actually devote to the process is a pretty mixed bag and both probably just want to finish quickly so they can get back to their actual job.
Long story short though all this really amounts to is that you shouldn't take the conclusions of a publication are face value, even a scientific one. I dont think this is evidence that science is failing somehow, the reality will never be that every individual study is 100% reliable...that is just unrealistic. Scientific truths dont emerge from individual studies...they emerge from countless studies converging on the same answer until the entire field accepts them as truth. I think the public going to individual studies for what to believe instead of the overall consensus of the scientific community is the mistake.8 -
Aaron_K123 wrote: »NorthCascades wrote: ».
Long story short though all this really amounts to is that you shouldn't take the conclusions of a publication are face value, even a scientific one. I dont think this is evidence that science is failing somehow, the reality will never be that every individual study is 100% reliable...that is just unrealistic. Scientific truths dont emerge from individual studies...they emerge from countless studies converging on the same answer until the entire field accepts them as truth. I think the public going to individual studies for what to believe instead of the overall consensus of the scientific community is the mistake.
Oh, I would agree there's not evidence science is failing. I think it's more a problem with the current system as it has been slowly changing in the last decade or more. Not the system of research in terms of a single study, but the system as a whole, with what research is funded, and how, and how politics and funding are shaping research and publishing.
As you said, scientific truths aren't found in one study. But currently, we're having some real problems with actually replicating the studies that are out there (https://www.bbc.com/news/science-environment-39054778). Some interesting, if frustrating, looks at studies in the past few years are finding very few repeated studies to see if the original study results were able to be replicated.
I would agree that relying on a single study is a mistake, but it's one that is also encouraged by much of the layman's literature that repeat the findings of one study as though it is some newly discovered absolute truth. And then, if it's disproven, it simply reinforces worries about the unreliability of science, when in fact, it's more the 'misunderstanding' of science by sources that are then passing along that to the general public.
...and I hope this didn't get incoherent here, as it is so late I'm not really sure what I just wrote, LOL.1 -
Oh, I would agree there's not evidence science is failing. I think it's more a problem with the current system as it has been slowly changing in the last decade or more. Not the system of research in terms of a single study, but the system as a whole, with what research is funded, and how, and how politics and funding are shaping research and publishing.
As you said, scientific truths aren't found in one study. But currently, we're having some real problems with actually replicating the studies that are out there (https://www.bbc.com/news/science-environment-39054778). Some interesting, if frustrating, looks at studies in the past few years are finding very few repeated studies to see if the original study results were able to be replicated.
I would agree that relying on a single study is a mistake, but it's one that is also encouraged by much of the layman's literature that repeat the findings of one study as though it is some newly discovered absolute truth. And then, if it's disproven, it simply reinforces worries about the unreliability of science, when in fact, it's more the 'misunderstanding' of science by sources that are then passing along that to the general public.
...and I hope this didn't get incoherent here, as it is so late I'm not really sure what I just wrote, LOL.
I think I do get what you are saying I just don't think that studies not being reproducible is somehow just a current issue and not something that has always been true. I don't think that the peer review system is suddenly just now failing, it is just that people in the public thought it was better than it actually is and are just shocked to hear that. The time in which someone becomes aware of an issue is not necessarily the time in which the issue began.
The reason why individual studies are not treated as scientific truths within the scientific community is precisely because any individual study is likely to be flawed in some way and perhaps even critically flawed. It is only when there are mountains of studies all pointing to the same thing that you can feel some confidence that it is an accurate representation of reality because it is unlikely that ALL of the studies are somehow invalid and yet all point to the same thing.
Scientists don't point to one study as evidence that something is absolutely true...the general public does that and the media does that and personally I don't think that is a good thing to do. Honestly I think there are a lot of the lay public who champion science so hard that they end up overselling it as somehow unimpeachable or that a "peer reviewed study" is just going to be true. I'm worried that there is going to be a backlash against the scientific community not based on what the scientific community itself claimed about its truths but rather about what the champions of science in the public realm claim about scientific truths that end up being on rather shaky ground of individual studies.
I don't know, maybe not perhaps I'm being over dramatic there but I am not really that shocked to hear that many studies out there cannot be reproduced.3 -
Aaron_K123 wrote: »Oh, I would agree there's not evidence science is failing. I think it's more a problem with the current system as it has been slowly changing in the last decade or more. Not the system of research in terms of a single study, but the system as a whole, with what research is funded, and how, and how politics and funding are shaping research and publishing.
As you said, scientific truths aren't found in one study. But currently, we're having some real problems with actually replicating the studies that are out there (https://www.bbc.com/news/science-environment-39054778). Some interesting, if frustrating, looks at studies in the past few years are finding very few repeated studies to see if the original study results were able to be replicated.
I would agree that relying on a single study is a mistake, but it's one that is also encouraged by much of the layman's literature that repeat the findings of one study as though it is some newly discovered absolute truth. And then, if it's disproven, it simply reinforces worries about the unreliability of science, when in fact, it's more the 'misunderstanding' of science by sources that are then passing along that to the general public.
...and I hope this didn't get incoherent here, as it is so late I'm not really sure what I just wrote, LOL.
I think I do get what you are saying I just don't think that studies not being reproducible is somehow just a current issue and not something that has always been true. I don't think that the peer review system is suddenly just now failing, it is just that people in the public thought it was better than it actually is and are just shocked to hear that. The time in which someone becomes aware of an issue is not necessarily the time in which the issue began.
The reason why individual studies are not treated as scientific truths within the scientific community is precisely because any individual study is likely to be flawed in some way and perhaps even critically flawed. It is only when there are mountains of studies all pointing to the same thing that you can feel some confidence that it is an accurate representation of reality because it is unlikely that ALL of the studies are somehow invalid and yet all point to the same thing.
Scientists don't point to one study as evidence that something is absolutely true...the general public does that and the media does that and personally I don't think that is a good thing to do. Honestly I think there are a lot of the lay public who champion science so hard that they end up overselling it as somehow unimpeachable or that a "peer reviewed study" is just going to be true. I'm worried that there is going to be a backlash against the scientific community not based on what the scientific community itself claimed about its truths but rather about what the champions of science in the public realm claim about scientific truths that end up being on rather shaky ground of individual studies.
I don't know, maybe not perhaps I'm being over dramatic there but I am not really that shocked to hear that many studies out there cannot be reproduced.
It's the volume of data at play here. A loose application of Price's Law and the Pareto Principle. Quality is sacrificed for volume.
I would be interested in seeing how this plays out in various disciplines. I would expect most physical science papers to be reproducible due to the abstract nature. Life/environmental sciences would be far less, and social sciences would be laughable.0 -
I’ve reached that age where my starting point before reading anything is “this is probably nonsense, but here goes...”.
Always ask yourself how the writer or publisher benefits from saying what they’re saying.2 -
I always read this stuff with a ton of skepticism.
The questions I ask myself.
1. What are they selling? They're almost ALWAYS selling something.
2. What is their opinion on eggs? Remember when eggs were bad? Then good? Then bad again? Then good. Then bad...
3. I need to see the actual scientific studies. Just referencing some vague unknowable study isn't enough. The actual study needs to be linked or I presume it's nonsense. Just like statistics, studies can be very biased and poorly conducted.
One of my favorite past times used to be reading Women's Weekly. I know, trash right? But what amused me is every week they'd have a different stance on what helps your thyroid ( I have a thyroid condition so it interests me).
Week one cauliflower is terrible for your thyroid! Avoid it at all costs!
Week two cauliflower is GREAT for your thyroid! Go ahead and eat it.
Week three cauliflower is horrible for thyroid problems! Don't eat it.
Week four the verdict is out. Nobody knows if cauliflower is good or bad.
It's such nonsense.2 -
yes. most of my friends jump on bandwagons. i try not to shake my head at them0
-
Aaron_K123 wrote: »NorthCascades wrote: »...When you read things, especially health and diet related, do you have a healthy sense of skepticism?
Oh heck yeah. Although I tend to have skepticism of claims, but also certain aspects of research as well. I am one of those that checks for conflicts of interest when a study is completed, for the agenda of whoever is reporting the claims, of the potential for bias in studies.
And while big established groups do often tend to have more well done research and I might often view them with more trust, they can also be more influenced by those with wealth and power, with 'the need to publish,' because the bigger groups tend to do studies that involve more potential gains in wealth by certain parties. There can also be more pressure on scientists to have findings that are the results their funding entity wants. And that same pressure can extend to those in the scientific publishing industry
I really respect scientists, and what they do. But a number of famous scientists or people in the know have spoken up in the past few years about how broken the science publishing system is currently, to the point that it has put into question many studies, which is REALLY a shame, as there is some really important stuff we're discovering, and without trust in the system, then people feel justified in just saying 'that's fake news,' to whatever they don't like. *
* a couple of the complaints/problems about the publishing system.
- Article from 'The science council and institute of health' - https://www.acsh.org/news/2015/05/19/science-publication-is-hopelessly-compromised-say-journal-editors
- article in scientist.com - https://www.the-scientist.com/uncategorized/is-peer-review-broken-47872
I am a scientist and have been on both sides of peer review...both having my work reviewed for publication and reviewing others work for publication. It certainly is not a perfect system and there is certainly room for bias and laziness amongst reviewers and room for shoddy work to slip by.
I think the public assumes peer review is a more rigorous process than it actually is. I get the sense that people think peer review means that the experts in a field attempt to reproduce the work or something and that certainly isn't the case. Frankly there is way way to much work put up for publication than there are people who have the time to do a real in depth critical review.
Most times what happens is a paper is submitted to a journal, that journal has an editor who is familiar with the field that the journal specializes in and sends that manuscript out to usually three reviewers. These reviewers dont work for the journal, they are just other scientists who work in that field. Given that fields are so specialized often a manuscript can end up in the hands of a reviewer that is only moderately familiar with the field. These reviewers have full time jobs unrelated to reviewing and they dont get any extra time or pay for the act of reviewing. So really the best you can expect is for them to read the entire manuscript armed with the knowledge of the field they have which might not be all that detailed and then just provide critical commentary. That goes back to the editor who decides whether to reject the paper, accept the paper or go back to the authors and ask them to address the reviewers criticisms which can range from not liking a phrasing of a sentence to requesting that an entire new experiment be performed.
All the reviewers have to go off of is what the authors say they did so there has to be some level of trust that the authors are correctly representing their work. Also any one person, even an expert in their field, is unlikely to know everything and can certainly miss dire problems with the work. For example a group might have a statistician who could spot an overly small sample size a mile away but the person from that group reviewing the paper isnt the statistician it's a biologist who can interpret the method used but may not recognize an issue with the sample size. Or maybe the statistician is the reviewer and can catch the issue with the sample size but doesn't catch that the buological method that was used has serious problems.
It is to much to ask of any given reviewer to put in a ton of time to study up on all the specifics and history of the field that the paper they are reviewing relates to. I mean typically speaking a reviewer probably is going to spend maybe half a day on it... not weeks.
Important to realize how many papers are published daily and how many many more are submitted for publication. There is no reward for doing a good job as a paper reviewer and no real punishment for doing a bad job. There is, however, a lot of pressure to finish the review to get back to your actual job. Not sure I see a way around this.
The best analogy I can think of for the peer review system outside of science is jury duty. Science publications are peer reviewed and defendants have a jury of their peers. There aren't professional jurors anymore than there are professional reviewers...they are basically drafted. The quality of "peer" you get and how much time they are going to actually devote to the process is a pretty mixed bag and both probably just want to finish quickly so they can get back to their actual job.
Long story short though all this really amounts to is that you shouldn't take the conclusions of a publication are face value, even a scientific one. I dont think this is evidence that science is failing somehow, the reality will never be that every individual study is 100% reliable...that is just unrealistic. Scientific truths dont emerge from individual studies...they emerge from countless studies converging on the same answer until the entire field accepts them as truth. I think the public going to individual studies for what to believe instead of the overall consensus of the scientific community is the mistake.
As a natural skeptic, I am enjoying this train of thought. I think this piece is very even-handed. I applaud the fact that, as a scientist, you are not holding up peer review as the ultimate authority, nor are you denigrating it as meaningless.
Having served on a jury recently in a murder trial, your reference to the "professional" side of that is interesting. As much as the lawyers and judge spoke about the intricacies of the law, ultimately the judge reminded us that the law was basically ours to interpret. And of course there are 12 different perspectives and reasoning capabilities in that room.
I'm not fond of the "appeal to authority" logical fallacy that is not necessarily used by authors of published material, but by readers with an agenda. "Hey dude, it's 'settled' science. See this 'peer-reviewed' study?" But that is not to say that authority on a subject is meaningless.
That said, I think peer-review is a valid concept. It's an imperfect one, but it's better than a lot of other possible ways to help discern truth.0 -
Silentpadna wrote: »Aaron_K123 wrote: »NorthCascades wrote: »...When you read things, especially health and diet related, do you have a healthy sense of skepticism?
Oh heck yeah. Although I tend to have skepticism of claims, but also certain aspects of research as well. I am one of those that checks for conflicts of interest when a study is completed, for the agenda of whoever is reporting the claims, of the potential for bias in studies.
And while big established groups do often tend to have more well done research and I might often view them with more trust, they can also be more influenced by those with wealth and power, with 'the need to publish,' because the bigger groups tend to do studies that involve more potential gains in wealth by certain parties. There can also be more pressure on scientists to have findings that are the results their funding entity wants. And that same pressure can extend to those in the scientific publishing industry
I really respect scientists, and what they do. But a number of famous scientists or people in the know have spoken up in the past few years about how broken the science publishing system is currently, to the point that it has put into question many studies, which is REALLY a shame, as there is some really important stuff we're discovering, and without trust in the system, then people feel justified in just saying 'that's fake news,' to whatever they don't like. *
* a couple of the complaints/problems about the publishing system.
- Article from 'The science council and institute of health' - https://www.acsh.org/news/2015/05/19/science-publication-is-hopelessly-compromised-say-journal-editors
- article in scientist.com - https://www.the-scientist.com/uncategorized/is-peer-review-broken-47872
I am a scientist and have been on both sides of peer review...both having my work reviewed for publication and reviewing others work for publication. It certainly is not a perfect system and there is certainly room for bias and laziness amongst reviewers and room for shoddy work to slip by.
I think the public assumes peer review is a more rigorous process than it actually is. I get the sense that people think peer review means that the experts in a field attempt to reproduce the work or something and that certainly isn't the case. Frankly there is way way to much work put up for publication than there are people who have the time to do a real in depth critical review.
Most times what happens is a paper is submitted to a journal, that journal has an editor who is familiar with the field that the journal specializes in and sends that manuscript out to usually three reviewers. These reviewers dont work for the journal, they are just other scientists who work in that field. Given that fields are so specialized often a manuscript can end up in the hands of a reviewer that is only moderately familiar with the field. These reviewers have full time jobs unrelated to reviewing and they dont get any extra time or pay for the act of reviewing. So really the best you can expect is for them to read the entire manuscript armed with the knowledge of the field they have which might not be all that detailed and then just provide critical commentary. That goes back to the editor who decides whether to reject the paper, accept the paper or go back to the authors and ask them to address the reviewers criticisms which can range from not liking a phrasing of a sentence to requesting that an entire new experiment be performed.
All the reviewers have to go off of is what the authors say they did so there has to be some level of trust that the authors are correctly representing their work. Also any one person, even an expert in their field, is unlikely to know everything and can certainly miss dire problems with the work. For example a group might have a statistician who could spot an overly small sample size a mile away but the person from that group reviewing the paper isnt the statistician it's a biologist who can interpret the method used but may not recognize an issue with the sample size. Or maybe the statistician is the reviewer and can catch the issue with the sample size but doesn't catch that the buological method that was used has serious problems.
It is to much to ask of any given reviewer to put in a ton of time to study up on all the specifics and history of the field that the paper they are reviewing relates to. I mean typically speaking a reviewer probably is going to spend maybe half a day on it... not weeks.
Important to realize how many papers are published daily and how many many more are submitted for publication. There is no reward for doing a good job as a paper reviewer and no real punishment for doing a bad job. There is, however, a lot of pressure to finish the review to get back to your actual job. Not sure I see a way around this.
The best analogy I can think of for the peer review system outside of science is jury duty. Science publications are peer reviewed and defendants have a jury of their peers. There aren't professional jurors anymore than there are professional reviewers...they are basically drafted. The quality of "peer" you get and how much time they are going to actually devote to the process is a pretty mixed bag and both probably just want to finish quickly so they can get back to their actual job.
Long story short though all this really amounts to is that you shouldn't take the conclusions of a publication are face value, even a scientific one. I dont think this is evidence that science is failing somehow, the reality will never be that every individual study is 100% reliable...that is just unrealistic. Scientific truths dont emerge from individual studies...they emerge from countless studies converging on the same answer until the entire field accepts them as truth. I think the public going to individual studies for what to believe instead of the overall consensus of the scientific community is the mistake.
As a natural skeptic, I am enjoying this train of thought. I think this piece is very even-handed. I applaud the fact that, as a scientist, you are not holding up peer review as the ultimate authority, nor are you denigrating it as meaningless.
Having served on a jury recently in a murder trial, your reference to the "professional" side of that is interesting. As much as the lawyers and judge spoke about the intricacies of the law, ultimately the judge reminded us that the law was basically ours to interpret. And of course there are 12 different perspectives and reasoning capabilities in that room.
I'm not fond of the "appeal to authority" logical fallacy that is not necessarily used by authors of published material, but by readers with an agenda. "Hey dude, it's 'settled' science. See this 'peer-reviewed' study?" But that is not to say that authority on a subject is meaningless.
That said, I think peer-review is a valid concept. It's an imperfect one, but it's better than a lot of other possible ways to help discern truth.
Science itself is predicated on the idea that human senses and intuitions are hopelessly biased and flawed and our best means of getting at truth is to conduct observational experiments designed to take the human element out of the equation as much as possible.
Peer review itself is of course a human system by necessity and as such it would be miraculous if it wasnt biased and flawed. I actually doubt there are many scientists who would claim peer review is somehow unimpeachable....but I dont think anyone has come up with a better idea really. It is important to critically review work before it is published and it makes sense that the experts in the field be the ones to do it. But yeah...there is bias, there are mistakes sure and there is not enough time reasonably to truly vet a body of work that took an entire lab 4 years to produce over a cup of coffee with a paper printout.
I mean what do people expect...that every submitted publication will have the experimental work repeated by a third party to confirm reproducibility before publication is accepted? In the parlance of the internet....ain't nobody got time for that.1 -
Just looking through my email here is the last time I got a request to review a manuscript submitted for publication in case anyone is curious as to what that looks like. Not particularly formal...kind of like a "hey guy, want to do some work for us?" Redacted most of it so it isn't showing anything potentially confidential but figure you can still get the gist.
0 -
Aaron_K123 wrote: »Silentpadna wrote: »Aaron_K123 wrote: »NorthCascades wrote: »...When you read things, especially health and diet related, do you have a healthy sense of skepticism?
Oh heck yeah. Although I tend to have skepticism of claims, but also certain aspects of research as well. I am one of those that checks for conflicts of interest when a study is completed, for the agenda of whoever is reporting the claims, of the potential for bias in studies.
And while big established groups do often tend to have more well done research and I might often view them with more trust, they can also be more influenced by those with wealth and power, with 'the need to publish,' because the bigger groups tend to do studies that involve more potential gains in wealth by certain parties. There can also be more pressure on scientists to have findings that are the results their funding entity wants. And that same pressure can extend to those in the scientific publishing industry
I really respect scientists, and what they do. But a number of famous scientists or people in the know have spoken up in the past few years about how broken the science publishing system is currently, to the point that it has put into question many studies, which is REALLY a shame, as there is some really important stuff we're discovering, and without trust in the system, then people feel justified in just saying 'that's fake news,' to whatever they don't like. *
* a couple of the complaints/problems about the publishing system.
- Article from 'The science council and institute of health' - https://www.acsh.org/news/2015/05/19/science-publication-is-hopelessly-compromised-say-journal-editors
- article in scientist.com - https://www.the-scientist.com/uncategorized/is-peer-review-broken-47872
I am a scientist and have been on both sides of peer review...both having my work reviewed for publication and reviewing others work for publication. It certainly is not a perfect system and there is certainly room for bias and laziness amongst reviewers and room for shoddy work to slip by.
I think the public assumes peer review is a more rigorous process than it actually is. I get the sense that people think peer review means that the experts in a field attempt to reproduce the work or something and that certainly isn't the case. Frankly there is way way to much work put up for publication than there are people who have the time to do a real in depth critical review.
Most times what happens is a paper is submitted to a journal, that journal has an editor who is familiar with the field that the journal specializes in and sends that manuscript out to usually three reviewers. These reviewers dont work for the journal, they are just other scientists who work in that field. Given that fields are so specialized often a manuscript can end up in the hands of a reviewer that is only moderately familiar with the field. These reviewers have full time jobs unrelated to reviewing and they dont get any extra time or pay for the act of reviewing. So really the best you can expect is for them to read the entire manuscript armed with the knowledge of the field they have which might not be all that detailed and then just provide critical commentary. That goes back to the editor who decides whether to reject the paper, accept the paper or go back to the authors and ask them to address the reviewers criticisms which can range from not liking a phrasing of a sentence to requesting that an entire new experiment be performed.
All the reviewers have to go off of is what the authors say they did so there has to be some level of trust that the authors are correctly representing their work. Also any one person, even an expert in their field, is unlikely to know everything and can certainly miss dire problems with the work. For example a group might have a statistician who could spot an overly small sample size a mile away but the person from that group reviewing the paper isnt the statistician it's a biologist who can interpret the method used but may not recognize an issue with the sample size. Or maybe the statistician is the reviewer and can catch the issue with the sample size but doesn't catch that the buological method that was used has serious problems.
It is to much to ask of any given reviewer to put in a ton of time to study up on all the specifics and history of the field that the paper they are reviewing relates to. I mean typically speaking a reviewer probably is going to spend maybe half a day on it... not weeks.
Important to realize how many papers are published daily and how many many more are submitted for publication. There is no reward for doing a good job as a paper reviewer and no real punishment for doing a bad job. There is, however, a lot of pressure to finish the review to get back to your actual job. Not sure I see a way around this.
The best analogy I can think of for the peer review system outside of science is jury duty. Science publications are peer reviewed and defendants have a jury of their peers. There aren't professional jurors anymore than there are professional reviewers...they are basically drafted. The quality of "peer" you get and how much time they are going to actually devote to the process is a pretty mixed bag and both probably just want to finish quickly so they can get back to their actual job.
Long story short though all this really amounts to is that you shouldn't take the conclusions of a publication are face value, even a scientific one. I dont think this is evidence that science is failing somehow, the reality will never be that every individual study is 100% reliable...that is just unrealistic. Scientific truths dont emerge from individual studies...they emerge from countless studies converging on the same answer until the entire field accepts them as truth. I think the public going to individual studies for what to believe instead of the overall consensus of the scientific community is the mistake.
As a natural skeptic, I am enjoying this train of thought. I think this piece is very even-handed. I applaud the fact that, as a scientist, you are not holding up peer review as the ultimate authority, nor are you denigrating it as meaningless.
Having served on a jury recently in a murder trial, your reference to the "professional" side of that is interesting. As much as the lawyers and judge spoke about the intricacies of the law, ultimately the judge reminded us that the law was basically ours to interpret. And of course there are 12 different perspectives and reasoning capabilities in that room.
I'm not fond of the "appeal to authority" logical fallacy that is not necessarily used by authors of published material, but by readers with an agenda. "Hey dude, it's 'settled' science. See this 'peer-reviewed' study?" But that is not to say that authority on a subject is meaningless.
That said, I think peer-review is a valid concept. It's an imperfect one, but it's better than a lot of other possible ways to help discern truth.
Science itself is predicated on the idea that human senses and intuitions are hopelessly biased and flawed and our best means of getting at truth is to conduct observational experiments designed to take the human element out of the equation as much as possible.
Peer review itself is of course a human system by necessity and as such it would be miraculous if it wasnt biased and flawed. I actually doubt there are many scientists who would claim peer review is somehow unimpeachable....but I dont think anyone has come up with a better idea really. It is important to critically review work before it is published and it makes sense that the experts in the field be the ones to do it. But yeah...there is bias, there are mistakes sure and there is not enough time reasonably to truly vet a body of work that took an entire lab 4 years to produce over a cup of coffee with a paper printout.
I mean what do people expect...that every submitted publication will have the experimental work repeated by a third party to confirm reproducibility before publication is accepted? In the parlance of the internet....ain't nobody got time for that.
This is part of what I find refreshing. In my somewhat limited discussions with people on this topic, those appealing the most stringently to peer-reviewed sources aren't folks like you. It's folks who want information that validates things in accordance with existing biases.
Even with that said, I tend to want to see those "vetted" (even if not perfectly) sources to be convinced, more so than anecdotal stuff.
Keep it coming @Aaron_K123. Always enjoy your contributions to these threads.0
This discussion has been closed.
Categories
- All Categories
- 1.4M Health, Wellness and Goals
- 393.6K Introduce Yourself
- 43.8K Getting Started
- 260.3K Health and Weight Loss
- 175.9K Food and Nutrition
- 47.5K Recipes
- 232.5K Fitness and Exercise
- 431 Sleep, Mindfulness and Overall Wellness
- 6.5K Goal: Maintaining Weight
- 8.6K Goal: Gaining Weight and Body Building
- 153K Motivation and Support
- 8K Challenges
- 1.3K Debate Club
- 96.3K Chit-Chat
- 2.5K Fun and Games
- 3.8K MyFitnessPal Information
- 24 News and Announcements
- 1.1K Feature Suggestions and Ideas
- 2.6K MyFitnessPal Tech Support Questions