Welcome to Debate Club! Please be aware that this is a space for respectful debate, and that your ideas will be challenged here. Please remember to critique the argument, not the author.

Peer Review Process Doesn't Work!

2»

Replies

  • stevencloser
    stevencloser Posts: 8,911 Member
    I read all the info I can get where it is peer reviewed or not and just do my own peer reviewing so to speak.

    As others have posted peer views does not speak to the validity of the published info.

    If I read the same thoughts from 3 articles from 3 different authors rom 3 different parts of the world from 3 different decades or greater of time I will grade it to have 33% chance of being valid.

    Not how probabilities work. At all.



    @FunkyTobias that is how probabilities work for me and how I train my staff to view probabilities. Will you state how probabilities work in your world? Probabilities is the real subject of this thread after all.


    Three possibilities does not imply three equal probabilities. When I hear hoofbeats I don't think

    33% horse
    33% zebra
    33% unicorn


    I understand that 100% because in my case it would only mean 99% chance a horse was making the noise. :)

    My question was will you state how probabilities work in your world.

    In the real world, you don't decide the probability of something.
  • GaleHawkins
    GaleHawkins Posts: 8,160 Member
    I read all the info I can get where it is peer reviewed or not and just do my own peer reviewing so to speak.

    As others have posted peer views does not speak to the validity of the published info.

    If I read the same thoughts from 3 articles from 3 different authors rom 3 different parts of the world from 3 different decades or greater of time I will grade it to have 33% chance of being valid.

    Not how probabilities work. At all.



    @FunkyTobias that is how probabilities work for me and how I train my staff to view probabilities. Will you state how probabilities work in your world? Probabilities is the real subject of this thread after all.


    Three possibilities does not imply three equal probabilities. When I hear hoofbeats I don't think

    33% horse
    33% zebra
    33% unicorn


    I understand that 100% because in my case it would only mean 99% chance a horse was making the noise. :)

    My question was will you state how probabilities work in your world.

    They work in my world just like they do in the real world. See my post above.

    Your prior post assumed equal probabilities for each source. This is just wrong.

    Sorry you inferred something I did not intend to imply.

    Batting 330 will historically give one a 77% probability of being inducted into the Baseball Hall of Fame I do know so being right 33% of the time leads to success more often than not in life I find at the age of 65. I know it does when it comes to losing weight and keeping it off in my case.
  • auddii
    auddii Posts: 15,357 Member
    I read all the info I can get where it is peer reviewed or not and just do my own peer reviewing so to speak.

    As others have posted peer views does not speak to the validity of the published info.

    If I read the same thoughts from 3 articles from 3 different authors rom 3 different parts of the world from 3 different decades or greater of time I will grade it to have 33% chance of being valid.

    Not how probabilities work. At all.



    But it explains a lot. I have a friend who had a sub once that said a 50% chance of rain meant either it will rain our it won't.

    *blink*

    *kitten*
  • stevencloser
    stevencloser Posts: 8,911 Member
    Gale, tell me how the part where it says "If peer review is good at anything, it appears to be keeping unpopular ideas from being published." fits in with when you said that unreproducible studies are more often cited? More often than not, the studies everyone is talking about (and thus citing) are the ones that go against established knowledge (which happen to be often not reproducible and worthless).
  • EvgeniZyntx
    EvgeniZyntx Posts: 24,208 Member
    auddii wrote: »
    I read all the info I can get where it is peer reviewed or not and just do my own peer reviewing so to speak.

    As others have posted peer views does not speak to the validity of the published info.

    If I read the same thoughts from 3 articles from 3 different authors rom 3 different parts of the world from 3 different decades or greater of time I will grade it to have 33% chance of being valid.

    Not how probabilities work. At all.



    But it explains a lot. I have a friend who had a sub once that said a 50% chance of rain meant either it will rain our it won't.

    *blink*

    *kitten*

    #notwrong

    It will rain our it won't.
    In either case, the lotus tree will wither, like this debate.
  • GaleHawkins
    GaleHawkins Posts: 8,160 Member
    Gale, tell me how the part where it says "If peer review is good at anything, it appears to be keeping unpopular ideas from being published." fits in with when you said that unreproducible studies are more often cited? More often than not, the studies everyone is talking about (and thus citing) are the ones that go against established knowledge (which happen to be often not reproducible and worthless).

    Steve please quote the post you are asking about.
  • stevencloser
    stevencloser Posts: 8,911 Member
    What you yourself quoted in your OP.
  • GaleHawkins
    GaleHawkins Posts: 8,160 Member
    DYELB wrote: »
    Did you ever actually go to school and...like...read a book or anything?

    Just because 8 different people in 8 different states over a period of 8 months took their GED and passed doesn't mean you get an 80% on it automatically.

    You may be correct but we are getting read to find out soon and will be OK with an 80%.
  • GaleHawkins
    GaleHawkins Posts: 8,160 Member
    Gale, tell me how the part where it says "If peer review is good at anything, it appears to be keeping unpopular ideas from being published." fits in with when you said that unreproducible studies are more often cited? More often than not, the studies everyone is talking about (and thus citing) are the ones that go against established knowledge (which happen to be often not reproducible and worthless).

    Steve I did not say anything about "unreproducible studies are more often sited" that I can find. You will have to talk with the person that made that statement.
  • FunkyTobias
    FunkyTobias Posts: 1,776 Member
    Gale, tell me how the part where it says "If peer review is good at anything, it appears to be keeping unpopular ideas from being published." fits in with when you said that unreproducible studies are more often cited? More often than not, the studies everyone is talking about (and thus citing) are the ones that go against established knowledge (which happen to be often not reproducible and worthless).

    Steve I did not say anything about "unreproducible studies are more often sited" that I can find. You will have to talk with the person that made that statement.

    Sigh

    It was in your original post
    The “bad” papers that failed to replicate were, on average, cited far more often than the papers that did!

    If you can't be bothered to vet your sources, can you please actually read them before pasting them?
  • GaleHawkins
    GaleHawkins Posts: 8,160 Member
    Gale, tell me how the part where it says "If peer review is good at anything, it appears to be keeping unpopular ideas from being published." fits in with when you said that unreproducible studies are more often cited? More often than not, the studies everyone is talking about (and thus citing) are the ones that go against established knowledge (which happen to be often not reproducible and worthless).

    Steve I did not say anything about "unreproducible studies are more often sited" that I can find. You will have to talk with the person that made that statement.

    Sigh

    It was in your original post
    The “bad” papers that failed to replicate were, on average, cited far more often than the papers that did!

    If you can't be bothered to vet your sources, can you please actually read them before pasting them?

    If you will read the OP linked article you will see your mistake. There was nothing to vet. If you have any sources that supports the author's position or distract from the author's position feel free to add to the subject of the debate.
  • JeromeBarry1
    JeromeBarry1 Posts: 10,182 Member
    I began to doubt the peer-review process when I learned that peer reviewers are mostly college professors with full teaching, researching, and grant-writing plates who don't have the apparatus or inclination to check the data, methods, or conclusions of the original writers.
  • FunkyTobias
    FunkyTobias Posts: 1,776 Member
    Gale, tell me how the part where it says "If peer review is good at anything, it appears to be keeping unpopular ideas from being published." fits in with when you said that unreproducible studies are more often cited? More often than not, the studies everyone is talking about (and thus citing) are the ones that go against established knowledge (which happen to be often not reproducible and worthless).

    Still waiting for @GaleHawkins to answer this
  • _John_
    _John_ Posts: 8,642 Member
    I began to doubt the peer-review process when I learned that peer reviewers are mostly college professors with full teaching, researching, and grant-writing plates who don't have the apparatus or inclination to check the data, methods, or conclusions of the original writers.

    well, if it makes you feel any better, the post-docs and grad students in their lab may have done it since the professor is so busy...
  • stealthq
    stealthq Posts: 4,298 Member
    edited April 2016
    I began to doubt the peer-review process when I learned that peer reviewers are mostly college professors with full teaching, researching, and grant-writing plates who don't have the apparatus or inclination to check the data, methods, or conclusions of the original writers.

    How exactly would anyone be expected to check any of those things (for clinical trials in particular), regardless of the apparatus they have on hand or their inclination?

    Reviewers can't be granted access to the full clinical data (HIPAA violation), so if anything in the paper required any of the data that can't be released, no verification can be done.

    Neither the journal nor the reviewers see the full experimental data unless it is required to be publicly released. Even then, only data referenced in the paper must be released. Have data that counters the paper's premise? Don't have to share that. Omics data generally is released. The rest is not. Heck, for radioactive gels and the like you don't even see the original in the paper, but a cropped and re-assembled picture/scan.

    Without access to the original biological samples, how would the reviewer even know the data was properly produced? Even WITH those samples, a fair amount of the data currently used in these papers is not reproducible in the way you may be thinking. RNASeq, for example, is a sampling technology. I can run it and get one sampling. You run with the same samples, same machine, same technician, same settings and get a different sampling. The samplings have a good probability of being similar, but there's no guarantee. They can be wildly different.

    There just is no way. It is a shame, but this is not physics or computer science where you can provide the info and if you are accurate and thorough it's guaranteed that the next person is able to reproduce what you've done.

    It is true that most reviewers are college professors. They also submit a good portion of the research, so ...

    Those of us who have non-academic jobs find it hard to find the time to review in addition to our regular jobs where our bosses don't give two hoots if we review papers, whereas being a reviewer is often considered a positive thing in academia.

    Personally, I find it more annoying that papers that are part of new fields of research often get reviewed by people who have no basis in that field. That's also not fixable - no qualified reviewers available, then you have to make do. But it is annoying to have one's paper stalled because of a couple of questions made in ignorance.