Blog

Why Reviewer Bias Still Shapes Which Papers Get Published

QED Science

Key Takeaways

  • Identity moves recommendations: Reviewers who could see a paper's prominent authors recommended acceptance more often than reviewers who read the same paper with the names removed.
  • Expectations move scrutiny: Reviewers given the null-result version of a paper found more planted errors and scored its identical Methods lower.
  • Blinding leaks: Double-blind review hides names only part of the time, and no review model hides the result.
  • Claims are the controllable part: What an author controls is how well each claim's evidence holds up for a reader who doesn't know who wrote it.

The same manuscript can earn a different review depending on whose name sits on the title page.

In a 2016 randomized trial at an orthopedics journal, 87% of reviewers recommended acceptance when they could see the paper's prominent authors, against 68% of reviewers who read it with the names removed. Nothing else about the manuscript changed.

For an author holding a harsh review, reviewer bias is hard to see from the inside. Each reviewer reads the paper once, and the author never sees the review a different name would have drawn. A rejection can reflect weak evidence, a surprising result, or the affiliation line, and the review rarely says which.

Rigorous authors can treat that bias as a measured effect and ask a narrower question. Which claims would hold up for a reader who doesn't know who wrote them?

Bias enters peer review at a few predictable points, and each review model blocks only part of it. Testing your own claims without your name attached can show which criticisms a missing piece of evidence would explain.

‍

How Reviewer Bias Enters the Peer Review Process

Reviewer bias is "the violation of impartiality in the evaluation of a submission," in the words of a 2013 review of peer review bias, where impartiality means every reviewer applies the same criteria in the same way. Bias needs a decision point to act on, and a typical journal submission passes through four:

  • Editor triage: An editor decides whether the manuscript goes out for review at all.
  • Reviewer selection: The editor chooses who reads the paper, sometimes from names the author suggested.
  • Reading and scoring: Each reviewer judges the manuscript, and identity cues and expectations about the result act here.
  • Editorial decision: The editor weighs reviews that may disagree and makes the call.

Grant review shows how much room individual judgment has at the scoring stage. In four mock NIH-style study sections, 43 oncology researchers reviewing a shared set of 25 eventually funded applications showed no agreement. Proposal writers can answer that spread by writing for specialist and generalist reviewers at once.

‍

Free for academic
researchers
Don’t wait for reviewers to find the gaps.
QED Science breaks your paper or grant into its core claims, and shows you exactly where the evidence holds up and where it doesn't.
Try QED Free

‍

Prestige, Affiliation, and Author Identity Effects

When reviewers can see who wrote a paper, prominent names and institutions can earn more acceptances for the same work. In the 2016 orthopedics trial, reviewers who saw the prominent names did not differ significantly from the masked group in how many planted errors they caught, yet they recommended acceptance more often. Single-blind peer review puts that cue in front of every reviewer.

A study of the American Heart Association's Scientific Sessions, a cardiology research meeting averaging 13,455 abstracts a year, points to the same prestige bias. Abstracts were reviewed with author names shown in 2000 and 2001 and blind from 2002 to 2004, and blinded review narrowed the gaps between US and non-US abstracts and between highly prestigious and nonprestigious US academic institutions. 

The study compared the two periods without randomization, so other changes over those years may also play a part. The table compares acceptance rates for each group under open and blinded review.

Abstract group Open review (2000-2001) Blinded review (2002-2004)
US authors 40.8% 33.4%
Non-US authors 22.6% 23.7%
Highly prestigious US academic institutions 51.3% 38.8%
Nonprestigious US academic institutions 32.6% 29.0%

‍

Confirmation Bias Against Unexpected Results

Reviewers who can't see the author can still see the result. Confirmation bias in research usually describes how scientists read their own data, and the same pull can affect reviewers who expect a finding to go one way.

In a 2010 randomized trial at two orthopedic journals, 210 reviewers each read one of two versions of a fabricated manuscript. The versions were identical except for the result, one positive and one showing no difference. Reviewers recommended publishing 97.3% of the positive versions and 80.0% of the no-difference versions. Reviewers also found more of the planted errors in the no-difference version, 0.85 on average against 0.41, and scored its Methods section lower, 7.53 against 8.24, though the two Methods sections were identical.

Rosalyn Yalow's 1977 Nobel lecture recounts showing in the mid-1950s that insulin-treated patients make insulin-binding antibodies, a concept "not acceptable to the immunologists" of the time. Science rejected the paper. The Journal of Clinical Investigation accepted it only after the authors dropped "insulin antibody" from the title and showed the binding globulin met a textbook definition of an antibody.

‍

Comparing Single-Blind, Double-Blind, and Open Review

Single-blind review hides the reviewers, double-blind review hides both sides, open review hides neither, and no model hides the result. The table compares the three models on who sees whose name, the bias each targets, what trials found, and where it leaks.

Model Who sees whose name Bias it targets What trials found Where it leaks
Single-blind Reviewers see the authors. Authors don't see the reviewers None by design Prominent authors drew more acceptance recommendations Every identity cue reaches the reviewer
Double-blind Neither side sees the other Author identity and prestige Lower acceptance rates than single-blind in pooled trials Reviewers judged masking successful only about half the time
Open Both sides see each other Reviewer accountability More invited reviewers declined, with no change in review quality Author identity stays visible

‍

A 2022 meta-analysis found that reviewers judged masking successful in only about 52% of cases across five randomized trials, and five trials covering 2,432 manuscripts put acceptance at 14.2% under double-blind peer review against 19.0% under single-blind. The trials varied widely. Choosing double-blind review where it's offered hides your name from some reviewers and the results from none.

At 25 Nature journals, 12% of 106,373 direct submissions from 2015 to 2017 chose double-blind review, with uptake at 4% for authors from top-10 institutions and 13% for those ranked outside the top 100. Those manuscripts fared worse at each stage, but authors chose the model themselves, so the study can't separate bias from quality.

Open peer review trades anonymity for accountability. In a 1999 BMJ trial, reviewers asked to sign their reports were more likely to decline the review, and review quality did not change. 

‍

Anonymized AI Assessment as a Bias Check

An AI read can hide who wrote a paper, but it still sees the result. Hiding names matters for a model too, since its recommendations can shift with the affiliation attached, though one JAMA test found a smaller shift than studies of human reviewers report.

In that 2024 JAMA study, GPT-3.5 produced 232,500 reviews of 30 medRxiv abstracts, each shown with one of 30 university affiliations or none. It accepted 38.4% with a top-tier affiliation and 36.7% with a low-tier one, a 1.7-point gap the authors say may not be practically important. Abstracts with no affiliation fared best, at 40.0%.

Anonymization leaves the field's expectations in place. A check that weighs each claim against prior work sets a higher evidence bar for a surprising result, and that bar shows an author what evidence a skeptical reviewer is likely to ask for.  A fair bar asks for more evidence of the surprising claim itself, while the 2010 reviewers went further and scored identical Methods lower, which no added evidence could fix. A useful pre-submission check, run by the author on their own manuscript, does four things, the same steps behind a quality score computed on anonymized manuscripts.

  • Strips identity first: Names and affiliations come off before anything is scored.
  • Scores each claim: It weighs each claim against its evidence, so a weak claim stands out from the rest.
  • Shows departures from prior work: A result that cuts against the literature is marked with the evidence it needs.
  • Shows its reasoning: It explains why it raised each finding, and the author decides what to change.

A check like this is a first-pass read to weigh alongside expert review. Its findings deserve the scrutiny an author gives any reviewer's comments.

‍

What Your Manuscript Looks Like Without Your Name on It

The lever left to an author is the manuscript itself. Before choosing a journal or review model, find the claim a skeptical reviewer would question first and see how it reads with your name removed. With a harsh review in hand, set its comments beside that anonymized read. Both criticisms likely point to missing evidence, and one the reviewer raised deserves a closer look.

QED Science removes author names and affiliations from every manuscript before scoring and evaluates each claim against its evidence. The same approach scored 57,455 bioRxiv preprints posted from May 2025 to April 2026 to select The 1%, the top 574. In QED's validation work, 12.9% of matched preprints that scored highly were later published in lower-ranked journals.

Pick the claim you expect a reviewer to question first, and the figure it rests on. Then run a pre-submission review and see what evidence the claim needs before a reviewer who doesn't know you reads it.

‍

FAQs

Does reviewer bias affect early career researchers more?

The evidence is mixed. In the 2016 orthopedics trial, well-known names drew more favorable recommendations, an edge newer authors are less likely to have. A study of 11,624 Canadian Institutes of Health Research grant applications found older applicants scored lower and past funding success raised scores. Double-blind review hides names, but reviewers judged the masking successful only about half the time. Checking each claim without your name attached is in every author's control.

Can authors request a different reviewer?

Before review, many journals let authors name people to exclude. Nature Portfolio journals accept requests to exclude one or two individuals or laboratories and usually honor them, though the editor's choice of referees is final. After reviews come back, write to the handling editor, name the comment you dispute, and ask whether another opinion is possible.

How do journals detect biased reviews?

Mostly through editors. A handling editor reads each review against the manuscript and the other reviews, and a comment targeting the authors stands out. At the ASBMB journals, editors also receive summaries of the gender and geography of the reviewers they invite, to catch skewed patterns. Journals that follow COPE guidance publish a process for complaints, including complaints about reviewers.

Can reviewer bias be grounds for an appeal?

It can, when you can point to it. A strong appeal quotes the specific review comment, shows how it misreads the data or rests on something other than the science, and answers every scientific point calmly. Contact the handling editor first, as Nature Portfolio's appeals policy asks.

Should I suggest reviewers when submitting a paper?

Yes, when the journal asks, though the editor will weigh your list. A study across 10 biomedical journals and a later study of three journals found that author-suggested reviewers wrote reviews as high-quality as editor-chosen ones and recommended acceptance more often. Suggest people with the right expertise and no conflict of interest, which rules out recent coauthors and close colleagues.

Free for academic
researchers
Don’t wait for reviewers to find the gaps.
QED Science breaks your paper or grant into its core claims, and shows you exactly where the evidence holds up and where it doesn't.
Try QED Free
Free for academic
researchers
Your work deserves more than hope.
Upload your first paper or grant and see what QED uncovers for you.
No credit card. No commitment. Secure and private.
Try QED Free
No items found.

Free access for academic researchers

Create your free QED account to validate your research, strengthen grant proposals, and uncover scientific insights.
We've sent you an access link.
Please check your inbox.

Didn't get your email? Check your spam folder or reach out to info@qedscience.com

Oops! Something went wrong while submitting the form.
Looking for QED for pharma, biotech or life science organizations?