Welcome to Debate Club! Please be aware that this is a space for respectful debate, and that your ideas will be challenged here. Please remember to critique the argument, not the author.

Peer Review Process Doesn't Work!

2»

Replies

  • FunkyTobias
    FunkyTobias Posts: 1,776 Member
    GaleHawkins wrote: »
    stevencloser wrote: »
    Gale, tell me how the part where it says "If peer review is good at anything, it appears to be keeping unpopular ideas from being published." fits in with when you said that unreproducible studies are more often cited? More often than not, the studies everyone is talking about (and thus citing) are the ones that go against established knowledge (which happen to be often not reproducible and worthless).

    Steve I did not say anything about "unreproducible studies are more often sited" that I can find. You will have to talk with the person that made that statement.

    Sigh

    It was in your original post
    GaleHawkins wrote: »
    The “bad” papers that failed to replicate were, on average, cited far more often than the papers that did!

    If you can't be bothered to vet your sources, can you please actually read them before pasting them?
  • GaleHawkins
    GaleHawkins Posts: 8,159 Member
    FunkyTobias wrote: »
    GaleHawkins wrote: »
    stevencloser wrote: »
    Gale, tell me how the part where it says "If peer review is good at anything, it appears to be keeping unpopular ideas from being published." fits in with when you said that unreproducible studies are more often cited? More often than not, the studies everyone is talking about (and thus citing) are the ones that go against established knowledge (which happen to be often not reproducible and worthless).

    Steve I did not say anything about "unreproducible studies are more often sited" that I can find. You will have to talk with the person that made that statement.

    Sigh

    It was in your original post
    GaleHawkins wrote: »
    The “bad” papers that failed to replicate were, on average, cited far more often than the papers that did!

    If you can't be bothered to vet your sources, can you please actually read them before pasting them?

    If you will read the OP linked article you will see your mistake. There was nothing to vet. If you have any sources that supports the author's position or distract from the author's position feel free to add to the subject of the debate.
  • JeromeBarry1
    JeromeBarry1 Posts: 10,179 Member
    I began to doubt the peer-review process when I learned that peer reviewers are mostly college professors with full teaching, researching, and grant-writing plates who don't have the apparatus or inclination to check the data, methods, or conclusions of the original writers.
  • FunkyTobias
    FunkyTobias Posts: 1,776 Member
    stevencloser wrote: »
    Gale, tell me how the part where it says "If peer review is good at anything, it appears to be keeping unpopular ideas from being published." fits in with when you said that unreproducible studies are more often cited? More often than not, the studies everyone is talking about (and thus citing) are the ones that go against established knowledge (which happen to be often not reproducible and worthless).

    Still waiting for @GaleHawkins to answer this
  • _John_
    _John_ Posts: 8,646 Member
    JeromeBarry1 wrote: »
    I began to doubt the peer-review process when I learned that peer reviewers are mostly college professors with full teaching, researching, and grant-writing plates who don't have the apparatus or inclination to check the data, methods, or conclusions of the original writers.

    well, if it makes you feel any better, the post-docs and grad students in their lab may have done it since the professor is so busy...
  • stealthq
    stealthq Posts: 4,298 Member
    edited April 2016
    JeromeBarry1 wrote: »
    I began to doubt the peer-review process when I learned that peer reviewers are mostly college professors with full teaching, researching, and grant-writing plates who don't have the apparatus or inclination to check the data, methods, or conclusions of the original writers.

    How exactly would anyone be expected to check any of those things (for clinical trials in particular), regardless of the apparatus they have on hand or their inclination?

    Reviewers can't be granted access to the full clinical data (HIPAA violation), so if anything in the paper required any of the data that can't be released, no verification can be done.

    Neither the journal nor the reviewers see the full experimental data unless it is required to be publicly released. Even then, only data referenced in the paper must be released. Have data that counters the paper's premise? Don't have to share that. Omics data generally is released. The rest is not. Heck, for radioactive gels and the like you don't even see the original in the paper, but a cropped and re-assembled picture/scan.

    Without access to the original biological samples, how would the reviewer even know the data was properly produced? Even WITH those samples, a fair amount of the data currently used in these papers is not reproducible in the way you may be thinking. RNASeq, for example, is a sampling technology. I can run it and get one sampling. You run with the same samples, same machine, same technician, same settings and get a different sampling. The samplings have a good probability of being similar, but there's no guarantee. They can be wildly different.

    There just is no way. It is a shame, but this is not physics or computer science where you can provide the info and if you are accurate and thorough it's guaranteed that the next person is able to reproduce what you've done.

    It is true that most reviewers are college professors. They also submit a good portion of the research, so ...

    Those of us who have non-academic jobs find it hard to find the time to review in addition to our regular jobs where our bosses don't give two hoots if we review papers, whereas being a reviewer is often considered a positive thing in academia.

    Personally, I find it more annoying that papers that are part of new fields of research often get reviewed by people who have no basis in that field. That's also not fixable - no qualified reviewers available, then you have to make do. But it is annoying to have one's paper stalled because of a couple of questions made in ignorance.

Categories