Key Takeaways
- Criteria come first: Reviewers fix eligibility criteria in the protocol before the search runs, and the population, intervention, and comparator in the review question supply most of them.
- Two screeners catch more: Cochrane requires two independent reviewers, and in one randomized trial, single-reviewer screening missed 13% of relevant studies, compared with 3% for dual screening.
- Every exclusion needs a reason: At full text, each excluded study gets a recorded reason that readers can trace through the flow diagram.
- AI screening needs a human in charge: Published evaluations show high sensitivity for AI screening tools in some configurations, and current guidance calls for human oversight and requires reporting whenever a tool makes or suggests eligibility judgments.
- Eligible is a lower bar than reliable: A study that passes screening still needs its claims and evidence tested before it shapes the synthesis.
Single-reviewer abstract screening missed 13% of relevant studies in a randomized trial of the practice.
Screening decides which studies a systematic review can see. A study excluded at the abstract stage never reaches data extraction, never enters the meta-analysis, and doesn’t shape the conclusion. Readers see the included studies in full, while most abstract-stage exclusions appear only as a count in the flow diagram.
That’s because output has grown fast. An estimated 80 systematic reviews a day were indexed in MEDLINE in 2019, more than 20 times the rate in 2000. The same stakes apply whether a review informs a clinical guideline, a grant application, or a pharma R&D program.
Rigorous review teams treat screening as a method with a measurable error rate. They write the criteria before they search, screen in pairs, measure how often they agree, and record every exclusion.
Each of those habits guards a stage where relevant studies can slip away, and AI tools are now moving into several of those stages.
What Evidence Screening Is and Where It Fits in a Systematic Review
Evidence screening is the step that turns search results into a set of included studies. It sits between the search and data extraction in the evidence synthesis workflow. Reviewers compare each record against the eligibility criteria in the protocol and decide whether to move it forward.
Screening usually runs in two passes:
- Reviewers first read titles and abstracts and remove records that clearly do not meet the criteria.
- They then retrieve the full text of everything that remains and make the final inclusion decision.
Most of what a search returns gets discarded. An analysis of 195 PROSPERO-registered reviews of medical interventions completed by 2014 found that, on average, only 2.94% of the records retrieved were included, and estimated a mean of 67.3 weeks from project start to publication.
Cochrane treats selection as a two-person job. Its methods standards require at least two people to work independently to decide whether each study meets the criteria, with a process for resolving disagreements defined in advance. Cochrane states that duplication reduces both mistakes and the chance that one person's biases shape selection.
researchers
How the PICO Framework Shapes Inclusion and Exclusion Criteria
The PICO framework turns a review question into rules a screener can apply. Cochrane's handbook notes that the population, interventions, and comparators in a review question usually translate directly into eligibility criteria. Study design is usually added as a further criterion.
The table below shows how this works for a hypothetical review of statins for primary prevention in adults with type 2 diabetes.
Outcomes are the part of PICO that usually falls outside the inclusion and exclusion criteria. Screening on reported outcomes would drop trials that measured an outcome but reported it poorly, and Cochrane's standards require including studies whether or not their outcome data are reported in a usable way to limit bias from selective reporting.
Teams write their inclusion and exclusion criteria into the protocol before the search runs. Cochrane also asks authors to define study design by its features, such as how participants were allocated to groups, instead of by the label a paper gives itself. When two screeners read the same abstract, precise criteria give them the same rule to apply.
How Title and Abstract Screening Works in Practice
Title and abstract screening is a quick pass through the full search results. Each screener reads a record, applies the criteria, and marks it as include, exclude, or unsure. Records marked unsure move to full-text review with included records, since an abstract may omit the detail needed for a firm decision.
The case for two screeners at this stage rests on direct evidence. In a crowd-based randomized trial on the Cochrane Crowd platform, single-reviewer screening missed 13% of relevant studies and dual screening missed 3%. The 2,000 abstracts in that trial came from two published reviews, one pharmacological and one in public health.
Good screening practice rests on four habits:
- Pilot the criteria: Both screeners apply the criteria to the same small sample first, then tighten any wording that produced disagreements.
- Resolve conflicts by a set rule: Disagreements go to discussion or a third reviewer, following a process written down before screening starts.
- Measure agreement: Teams track agreement between screeners, usually with Cohen's kappa. Cochrane's rapid review guidance lets teams move to single screening after two reviewers screen at least 20% of records and agreement is high, such as a kappa of 0.8 or above. The authors describe the 20% figure as experience-based.
- Log every decision: Screening software records each inclusion and exclusion, and those counts later fill in the flow diagram.
What Happens at the Full-Text Review Stage and How Exclusions Are Documented
Full-text review applies the same criteria with the whole paper in hand. Reviewers also decide details abstracts often leave out, such as how participants were allocated, what the comparator arm received, how long follow-up lasted, and whether the population matches the protocol.
A single trial may appear as a conference abstract, a main paper and a later follow-up analysis. Cochrane's standards require reviewers to collate multiple reports of the same study and to document the selection process in enough detail to build a flow diagram and a table of excluded studies. PRISMA 2020 adds that authors should cite studies that might appear to meet the inclusion criteria but were excluded, and explain why.
A usable exclusion reason names the criterion that failed. "Wrong comparator: statin versus ezetimibe" tells a reader exactly why a trial is missing. "Not relevant" tells them nothing and leaves the decision impossible to check.
How AI Tools Are Changing Evidence Screening in Systematic Reviews
Title and abstract screening is the stage of evidence synthesis with the most published evidence behind AI tools. A team deciding whether to use them needs to know what the evaluations show, where they fall short, and what current guidance requires.
What the Evidence Shows
A 2026 review of reviews identified 46 tools across 21 prior reviews. Of these, 10 reported sensitivity for AI-powered title and abstract screening, and all 10 reached at least 95% in at least one configuration. The authors concluded that the evidence supports human-supervised automation of that stage and remains limited for other review tasks.
One common approach is active learning, where the tool re-ranks unscreened records after each decision so likely records surface first. In retrospective simulations on four labeled datasets, including one from software engineering, the ASReview tool found 95% of relevant records after 8% to 33% of records had been screened.
Where the Evidence Falls Short
Performance can fall well short of those figures. When researchers tested two large language models on a single cardiovascular review of 984 records, the best configurations reached 77.5% sensitivity for Llama 3 and 56.2% for GPT-4o mini, and the authors concluded that manual validation remains necessary.
Sensitivity matters most in screening, because a study excluded at this stage drops out of every later step. Precision decides how much reading time the tool saves. The same two measures come up whenever researchers evaluate AI feedback, including work on separating genuine gaps from noise in manuscript review feedback.
What Current Guidance Requires
In November 2025, Cochrane, the Campbell Collaboration, JBI and the Collaboration for Environmental Evidence issued a joint position statement on responsible AI use. It holds that authors remain accountable for their review, including the decision to use AI, and that AI should be used with human oversight. Authors must report any tool that makes or suggests eligibility judgments.
Reporting standards already expect this. PRISMA 2020 asks authors to state how many reviewers screened each record, whether they worked independently, and which automation tools they used.
Test the Claims Inside the Studies You Include
Screening establishes that a study fits the review question. Whether its claims hold up is a separate question, and formal risk-of-bias assessment answers only part of it.
QED Science looks at the claims themselves. It breaks a paper into its individual claims and evaluates the evidence behind each one. Its QED Score then rates the whole paper on originality and validity, and it has been validated against journal rank and blinded expert judgment on life-science manuscripts. It doesn't replace risk-of-bias tools, and the review team keeps every inclusion decision.
Upload a paper you're reading and see where the evidence behind each claim holds up and where it's thin. To keep up with new work in your area between update searches, the Insights Feed ranks new PubMed and bioRxiv papers, down to single claims, based on your research profile and interests.
FAQs
researchers
researchers