Welcome to Debate Club! Please be aware that this is a space for respectful debate, and that your ideas will be challenged here. Please remember to critique the argument, not the author.

Peer Review Process Doesn't Work!

GaleHawkins
GaleHawkins Posts: 8,160 Member
theweek.com/articles/618141/big-science-broken

"The peer review process doesn't work. "

This is the above article source:
firstthings.com/article/2016/05/scientific-regress

"If peer review is good at anything, it appears to be keeping unpopular ideas from being published.

Consider the finding of another (yes, another) of these replicability studies, this time from a group of cancer researchers. In addition to reaching the now unsurprising conclusion that only a dismal 11 percent of the preclinical cancer research they examined could be validated after the fact, the authors identified another horrifying pattern: The “bad” papers that failed to replicate were, on average, cited far more often than the papers that did! As the authors put it, “some non-reproducible preclinical papers had spawned an entire field, with hundreds of secondary publications that expanded on elements of the original observation, but did not actually seek to confirm or falsify its fundamental basis.”"

If peer reviewed articles can be without merit what info can we depend on to make health decisions or any kind for that matter?

Just look at the numbers dying prematurely with heart disease, dementia, etc from lowering their cholesterol numbers based a false research results to only increase their risks of premature death.
«1

Replies

  • Christine_72
    Christine_72 Posts: 16,049 Member
    Meh, I take most "studies" with a grain of salt... Common sense has always guided me the right way.
  • stealthq
    stealthq Posts: 4,298 Member
    Peer review can only check that what is published is internally consistent and is reported to follow proper procedures.

    It can't check that the data is accurately reported or samples properly handled unless inconsistencies are in the paper and not left out in unreported experiments.

    Also, since the reviewers for biology research papers are usually other biologists, it doesn't enforce proper study design, because inadequate knowledge of statistics is common in that group.
  • Christine_72
    Christine_72 Posts: 16,049 Member
    Meh, I take most "studies" with a grain of salt... Common sense has always guided me the right way.

    Common sense is great, but it works better in my experience when I'm adding and evaluating new facts and ideas to go with it.

    Good point. Perhaps I should have said i don't take any studies as Gospel, and don't automatically believe every word I read..
  • sunnybeaches105
    sunnybeaches105 Posts: 2,831 Member
    edited April 2016
    I suspect that rather than discussing ways to correct some of the flaws in the process such as publishing null results (by those with the education and experience to examine them) that this thread will continue to go down the anti-science yellow brick road.

    ETA: oh, psychology . . . Yes, "big science" . . .
  • markrgeary1
    markrgeary1 Posts: 853 Member
    So the peer review process doesn't work. IK, let's just believe that 100%, so what does? If the best we have is no good, we should abandon it and do ? What? What?

    Oh it's the best way we know. Kinda like other things. Sounds like the author has an issue. IME Folks that complain about peer reviews generally have their work shredded by them.
  • T1DCarnivoreRunner
    T1DCarnivoreRunner Posts: 11,502 Member
    For one of my grad school classes, I cited a journal article with an entirely new method and a second journal article that criticized the first. In the same paper, I cited several other sources related to the topic (but with legacy approaches). In that case, the peer-review process worked. The first author contributed what was a new idea at the time. Someone else improved upon his idea with additional insight. That is how it is supposed to work.
  • GaleHawkins
    GaleHawkins Posts: 8,160 Member
    edited April 2016
    I read all the info I can get where it is peer reviewed or not and just do my own peer reviewing so to speak.

    As others have posted peer views does not speak to the validity of the published info.

    If I read the same thoughts from 3 articles from 3 different authors rom 3 different parts of the world from 3 different decades or greater of time I will grade it to have 33% chance of being valid. I was able to do this with the LCHF WOE over a period of > 100 years. Someone said there was nothing new under the sun and I am inclined to think so at the age of 65.
  • GaleHawkins
    GaleHawkins Posts: 8,160 Member
    cwolfman13 wrote: »
    theweek.com/articles/618141/big-science-broken

    "The peer review process doesn't work. "

    This is the above article source:
    firstthings.com/article/2016/05/scientific-regress

    "If peer review is good at anything, it appears to be keeping unpopular ideas from being published.

    Consider the finding of another (yes, another) of these replicability studies, this time from a group of cancer researchers. In addition to reaching the now unsurprising conclusion that only a dismal 11 percent of the preclinical cancer research they examined could be validated after the fact, the authors identified another horrifying pattern: The “bad” papers that failed to replicate were, on average, cited far more often than the papers that did! As the authors put it, “some non-reproducible preclinical papers had spawned an entire field, with hundreds of secondary publications that expanded on elements of the original observation, but did not actually seek to confirm or falsify its fundamental basis.”"

    If peer reviewed articles can be without merit what info can we depend on to make health decisions or any kind for that matter?

    Just look at the numbers dying prematurely with heart disease, dementia, etc from lowering their cholesterol numbers based a false research results to only increase their risks of premature death.

    And how was it actually determined that the lowering of their cholesterol contributed directly to their heart disease, dementia, and pre-mature death? Did they take into account that those individuals likely had other health issues that could and would have contributed to these issues?

    I find it interesting because most people who eat, say, something resembling the Mediterranean diet have pretty low LDL levels and good HDL levels...and they're not exactly dying pre-maturely from their lower cholesterol levels.

    Sounds to me like more LCHF propaganda fear mongering crap to me.

    @cwolfman13 why would you think people on the Mediterranean diet are even being given statin drugs that may increase the risk of premature death? Increase of premature dead risk that I have read in peer reviewed articles was from lowering cholesterol with drugs. If one can lower it by the Mediterranean or any other way of eating that should only lower the risks of premature death?
  • FunkyTobias
    FunkyTobias Posts: 1,776 Member
    I read all the info I can get where it is peer reviewed or not and just do my own peer reviewing so to speak.

    As others have posted peer views does not speak to the validity of the published info.

    If I read the same thoughts from 3 articles from 3 different authors rom 3 different parts of the world from 3 different decades or greater of time I will grade it to have 33% chance of being valid.

    Not how probabilities work. At all.



  • cwolfman13
    cwolfman13 Posts: 41,879 Member
    cwolfman13 wrote: »
    theweek.com/articles/618141/big-science-broken

    "The peer review process doesn't work. "

    This is the above article source:
    firstthings.com/article/2016/05/scientific-regress

    "If peer review is good at anything, it appears to be keeping unpopular ideas from being published.

    Consider the finding of another (yes, another) of these replicability studies, this time from a group of cancer researchers. In addition to reaching the now unsurprising conclusion that only a dismal 11 percent of the preclinical cancer research they examined could be validated after the fact, the authors identified another horrifying pattern: The “bad” papers that failed to replicate were, on average, cited far more often than the papers that did! As the authors put it, “some non-reproducible preclinical papers had spawned an entire field, with hundreds of secondary publications that expanded on elements of the original observation, but did not actually seek to confirm or falsify its fundamental basis.”"

    If peer reviewed articles can be without merit what info can we depend on to make health decisions or any kind for that matter?

    Just look at the numbers dying prematurely with heart disease, dementia, etc from lowering their cholesterol numbers based a false research results to only increase their risks of premature death.

    And how was it actually determined that the lowering of their cholesterol contributed directly to their heart disease, dementia, and pre-mature death? Did they take into account that those individuals likely had other health issues that could and would have contributed to these issues?

    I find it interesting because most people who eat, say, something resembling the Mediterranean diet have pretty low LDL levels and good HDL levels...and they're not exactly dying pre-maturely from their lower cholesterol levels.

    Sounds to me like more LCHF propaganda fear mongering crap to me.

    @cwolfman13 why would you think people on the Mediterranean diet are even being given statin drugs that may increase the risk of premature death? Increase of premature dead risk that I have read in peer reviewed articles was from lowering cholesterol with drugs. If one can lower it by the Mediterranean or any other way of eating that should only lower the risks of premature death?

    You didn't say statin drugs...you stated lowering cholesterol as a detriment to health. I used to have very high LDL and I lowered it from 170 to 93 with diet an exercise. All you noted was "lowering cholesterol" as if lower cholesterol was the risk factor.

    I'm personally not a fan of statin drugs, but I'm also not willing to say that these are direct contributors either...most people I know who have been prescribed statins have a lot of other stuff going on that would/could contribute to those health issues.
  • GaleHawkins
    GaleHawkins Posts: 8,160 Member
    cwolfman13 wrote: »
    cwolfman13 wrote: »
    theweek.com/articles/618141/big-science-broken

    "The peer review process doesn't work. "

    This is the above article source:
    firstthings.com/article/2016/05/scientific-regress

    "If peer review is good at anything, it appears to be keeping unpopular ideas from being published.

    Consider the finding of another (yes, another) of these replicability studies, this time from a group of cancer researchers. In addition to reaching the now unsurprising conclusion that only a dismal 11 percent of the preclinical cancer research they examined could be validated after the fact, the authors identified another horrifying pattern: The “bad” papers that failed to replicate were, on average, cited far more often than the papers that did! As the authors put it, “some non-reproducible preclinical papers had spawned an entire field, with hundreds of secondary publications that expanded on elements of the original observation, but did not actually seek to confirm or falsify its fundamental basis.”"

    If peer reviewed articles can be without merit what info can we depend on to make health decisions or any kind for that matter?

    Just look at the numbers dying prematurely with heart disease, dementia, etc from lowering their cholesterol numbers based a false research results to only increase their risks of premature death.

    And how was it actually determined that the lowering of their cholesterol contributed directly to their heart disease, dementia, and pre-mature death? Did they take into account that those individuals likely had other health issues that could and would have contributed to these issues?

    I find it interesting because most people who eat, say, something resembling the Mediterranean diet have pretty low LDL levels and good HDL levels...and they're not exactly dying pre-maturely from their lower cholesterol levels.

    Sounds to me like more LCHF propaganda fear mongering crap to me.

    @cwolfman13 why would you think people on the Mediterranean diet are even being given statin drugs that may increase the risk of premature death? Increase of premature dead risk that I have read in peer reviewed articles was from lowering cholesterol with drugs. If one can lower it by the Mediterranean or any other way of eating that should only lower the risks of premature death?

    You didn't say statin drugs...you stated lowering cholesterol as a detriment to health. I used to have very high LDL and I lowered it from 170 to 93 with diet an exercise. All you noted was "lowering cholesterol" as if lower cholesterol was the risk factor.

    I'm personally not a fan of statin drugs, but I'm also not willing to say that these are direct contributors either...most people I know who have been prescribed statins have a lot of other stuff going on that would/could contribute to those health issues.

    @cwolfman13 I should have been more clear earlier and I will try to be more careful.

    The one thing that statins can do that is helpful is to lower levels of inflammation in some people. It is the down the road concerns about statins that concerns me as in 20 years down the road for the guy that is 50 today.

    My take is lowering total cholesterol down to 200 is fine. Lower then that for me at the age of 65 I think research shows an increase of premature death for me. We all are dying but the when and how may be somewhat in our hands.
  • GaleHawkins
    GaleHawkins Posts: 8,160 Member
    I read all the info I can get where it is peer reviewed or not and just do my own peer reviewing so to speak.

    As others have posted peer views does not speak to the validity of the published info.

    If I read the same thoughts from 3 articles from 3 different authors rom 3 different parts of the world from 3 different decades or greater of time I will grade it to have 33% chance of being valid.

    Not how probabilities work. At all.



    @FunkyTobias that is how probabilities work for me and how I train my staff to view probabilities. Will you state how probabilities work in your world? Probabilities is the real subject of this thread after all.
  • GaleHawkins
    GaleHawkins Posts: 8,160 Member
    I read all the info I can get where it is peer reviewed or not and just do my own peer reviewing so to speak.

    As others have posted peer views does not speak to the validity of the published info.

    If I read the same thoughts from 3 articles from 3 different authors rom 3 different parts of the world from 3 different decades or greater of time I will grade it to have 33% chance of being valid.

    Not how probabilities work. At all.



    @FunkyTobias that is how probabilities work for me and how I train my staff to view probabilities. Will you state how probabilities work in your world? Probabilities is the real subject of this thread after all.


    Three possibilities does not imply three equal probabilities. When I hear hoofbeats I don't think

    33% horse
    33% zebra
    33% unicorn


    I understand that 100% because in my case it would only mean 99% chance a horse was making the noise. :)

    My question was will you state how probabilities work in your world.