# Calorie Counter

You are currently viewing the message boards in:

# Peer Review Process Doesn't Work!

2»

## Replies

• Posts: 8,917Member Member
I read all the info I can get where it is peer reviewed or not and just do my own peer reviewing so to speak.

As others have posted peer views does not speak to the validity of the published info.

If I read the same thoughts from 3 articles from 3 different authors rom 3 different parts of the world from 3 different decades or greater of time I will grade it to have 33% chance of being valid.

Not how probabilities work. At all.

@FunkyTobias that is how probabilities work for me and how I train my staff to view probabilities. Will you state how probabilities work in your world? Probabilities is the real subject of this thread after all.

Three possibilities does not imply three equal probabilities. When I hear hoofbeats I don't think

33% horse
33% zebra
33% unicorn

I understand that 100% because in my case it would only mean 99% chance a horse was making the noise.

My question was will you state how probabilities work in your world.

In the real world, you don't decide the probability of something.
• Posts: 7,623Member Member
I read all the info I can get where it is peer reviewed or not and just do my own peer reviewing so to speak.

As others have posted peer views does not speak to the validity of the published info.

If I read the same thoughts from 3 articles from 3 different authors rom 3 different parts of the world from 3 different decades or greater of time I will grade it to have 33% chance of being valid.

Not how probabilities work. At all.

@FunkyTobias that is how probabilities work for me and how I train my staff to view probabilities. Will you state how probabilities work in your world? Probabilities is the real subject of this thread after all.

Three possibilities does not imply three equal probabilities. When I hear hoofbeats I don't think

33% horse
33% zebra
33% unicorn

I understand that 100% because in my case it would only mean 99% chance a horse was making the noise.

My question was will you state how probabilities work in your world.

They work in my world just like they do in the real world. See my post above.

Your prior post assumed equal probabilities for each source. This is just wrong.

Sorry you inferred something I did not intend to imply.

Batting 330 will historically give one a 77% probability of being inducted into the Baseball Hall of Fame I do know so being right 33% of the time leads to success more often than not in life I find at the age of 65. I know it does when it comes to losing weight and keeping it off in my case.
• Posts: 15,410Member Member
I read all the info I can get where it is peer reviewed or not and just do my own peer reviewing so to speak.

As others have posted peer views does not speak to the validity of the published info.

If I read the same thoughts from 3 articles from 3 different authors rom 3 different parts of the world from 3 different decades or greater of time I will grade it to have 33% chance of being valid.

Not how probabilities work. At all.

But it explains a lot. I have a friend who had a sub once that said a 50% chance of rain meant either it will rain our it won't.

*kitten*
• Posts: 8,917Member Member
Gale, tell me how the part where it says "If peer review is good at anything, it appears to be keeping unpopular ideas from being published." fits in with when you said that unreproducible studies are more often cited? More often than not, the studies everyone is talking about (and thus citing) are the ones that go against established knowledge (which happen to be often not reproducible and worthless).
• Posts: 24,424Member Member
auddii wrote: »
I read all the info I can get where it is peer reviewed or not and just do my own peer reviewing so to speak.

As others have posted peer views does not speak to the validity of the published info.

If I read the same thoughts from 3 articles from 3 different authors rom 3 different parts of the world from 3 different decades or greater of time I will grade it to have 33% chance of being valid.

Not how probabilities work. At all.

But it explains a lot. I have a friend who had a sub once that said a 50% chance of rain meant either it will rain our it won't.

*kitten*

#notwrong

It will rain our it won't.
In either case, the lotus tree will wither, like this debate.
• Posts: 7,623Member Member
Gale, tell me how the part where it says "If peer review is good at anything, it appears to be keeping unpopular ideas from being published." fits in with when you said that unreproducible studies are more often cited? More often than not, the studies everyone is talking about (and thus citing) are the ones that go against established knowledge (which happen to be often not reproducible and worthless).

• Posts: 8,917Member Member
What you yourself quoted in your OP.
• Posts: 7,623Member Member
DYELB wrote: »
Did you ever actually go to school and...like...read a book or anything?

Just because 8 different people in 8 different states over a period of 8 months took their GED and passed doesn't mean you get an 80% on it automatically.

You may be correct but we are getting read to find out soon and will be OK with an 80%.
• Posts: 7,623Member Member
Gale, tell me how the part where it says "If peer review is good at anything, it appears to be keeping unpopular ideas from being published." fits in with when you said that unreproducible studies are more often cited? More often than not, the studies everyone is talking about (and thus citing) are the ones that go against established knowledge (which happen to be often not reproducible and worthless).

Steve I did not say anything about "unreproducible studies are more often sited" that I can find. You will have to talk with the person that made that statement.
• Posts: 1,776Member Member
Gale, tell me how the part where it says "If peer review is good at anything, it appears to be keeping unpopular ideas from being published." fits in with when you said that unreproducible studies are more often cited? More often than not, the studies everyone is talking about (and thus citing) are the ones that go against established knowledge (which happen to be often not reproducible and worthless).

Steve I did not say anything about "unreproducible studies are more often sited" that I can find. You will have to talk with the person that made that statement.

Sigh

It was in your original post
The “bad” papers that failed to replicate were, on average, cited far more often than the papers that did!

If you can't be bothered to vet your sources, can you please actually read them before pasting them?
• Posts: 7,623Member Member
Gale, tell me how the part where it says "If peer review is good at anything, it appears to be keeping unpopular ideas from being published." fits in with when you said that unreproducible studies are more often cited? More often than not, the studies everyone is talking about (and thus citing) are the ones that go against established knowledge (which happen to be often not reproducible and worthless).

Steve I did not say anything about "unreproducible studies are more often sited" that I can find. You will have to talk with the person that made that statement.

Sigh

It was in your original post
The “bad” papers that failed to replicate were, on average, cited far more often than the papers that did!

If you can't be bothered to vet your sources, can you please actually read them before pasting them?

If you will read the OP linked article you will see your mistake. There was nothing to vet. If you have any sources that supports the author's position or distract from the author's position feel free to add to the subject of the debate.
• Posts: 10,115Member Member
I began to doubt the peer-review process when I learned that peer reviewers are mostly college professors with full teaching, researching, and grant-writing plates who don't have the apparatus or inclination to check the data, methods, or conclusions of the original writers.
• Posts: 1,776Member Member
Gale, tell me how the part where it says "If peer review is good at anything, it appears to be keeping unpopular ideas from being published." fits in with when you said that unreproducible studies are more often cited? More often than not, the studies everyone is talking about (and thus citing) are the ones that go against established knowledge (which happen to be often not reproducible and worthless).

Still waiting for @GaleHawkins to answer this
• Posts: 8,601Member Member
I began to doubt the peer-review process when I learned that peer reviewers are mostly college professors with full teaching, researching, and grant-writing plates who don't have the apparatus or inclination to check the data, methods, or conclusions of the original writers.

well, if it makes you feel any better, the post-docs and grad students in their lab may have done it since the professor is so busy...
• Posts: 4,307Member Member
I began to doubt the peer-review process when I learned that peer reviewers are mostly college professors with full teaching, researching, and grant-writing plates who don't have the apparatus or inclination to check the data, methods, or conclusions of the original writers.

How exactly would anyone be expected to check any of those things (for clinical trials in particular), regardless of the apparatus they have on hand or their inclination?

Reviewers can't be granted access to the full clinical data (HIPAA violation), so if anything in the paper required any of the data that can't be released, no verification can be done.

Neither the journal nor the reviewers see the full experimental data unless it is required to be publicly released. Even then, only data referenced in the paper must be released. Have data that counters the paper's premise? Don't have to share that. Omics data generally is released. The rest is not. Heck, for radioactive gels and the like you don't even see the original in the paper, but a cropped and re-assembled picture/scan.

Without access to the original biological samples, how would the reviewer even know the data was properly produced? Even WITH those samples, a fair amount of the data currently used in these papers is not reproducible in the way you may be thinking. RNASeq, for example, is a sampling technology. I can run it and get one sampling. You run with the same samples, same machine, same technician, same settings and get a different sampling. The samplings have a good probability of being similar, but there's no guarantee. They can be wildly different.

There just is no way. It is a shame, but this is not physics or computer science where you can provide the info and if you are accurate and thorough it's guaranteed that the next person is able to reproduce what you've done.

It is true that most reviewers are college professors. They also submit a good portion of the research, so ...

Those of us who have non-academic jobs find it hard to find the time to review in addition to our regular jobs where our bosses don't give two hoots if we review papers, whereas being a reviewer is often considered a positive thing in academia.

Personally, I find it more annoying that papers that are part of new fields of research often get reviewed by people who have no basis in that field. That's also not fixable - no qualified reviewers available, then you have to make do. But it is annoying to have one's paper stalled because of a couple of questions made in ignorance.
edited April 2016