Resources - RESEARCH LITERACY
Systematic review vs meta-analysis: what is the difference?
A systematic review finds and appraises all studies on one question; a meta-analysis statistically combines their results. Many reviews include one, not all.
A systematic review is a structured search for, and appraisal of, all the studies that answer a specific question, using methods set out in advance to reduce bias. A meta-analysis is a statistical technique that combines the numerical results of two or more of those studies into one overall estimate. Many systematic reviews include a meta-analysis, but not all do, and a meta-analysis is only as good as the review behind it.
What is a systematic review?
The Cochrane Handbook, the methods manual used for Cochrane reviews, describes a systematic review as an attempt to gather all the evidence that meets eligibility criteria set in advance, in order to answer one specific question, using explicit methods chosen to minimise bias. The key word is systematic. The aim is to stop the reviewer's own preferences deciding which studies count.
In practice that means a written protocol, searches of several databases, clear rules about which studies qualify, an assessment of each study's risk of bias, and a transparent report of what was found and what was left out. The PRISMA 2020 statement, published in the BMJ by Matthew Page and colleagues, replaced the original 2009 guideline and sets out a 27-item checklist for reporting systematic reviews, along with flow diagrams showing how many studies were found, screened and excluded.
Why not just ask an expert?
Because experts can lag far behind the evidence. In a 1992 JAMA study, Elliott Antman, Joseph Lau and colleagues compared accumulating trial results for heart attack treatments with what clinical experts recommended in review articles and textbook chapters. Review articles often failed to mention important advances or were slow to recommend effective preventive measures. In some cases, experts went on recommending treatments that had no effect on mortality or were potentially harmful.
What is a meta-analysis?
The Cochrane Handbook defines meta-analysis as the statistical combination of results from two or more separate studies. Each study's result is weighted, so that more precise studies count for more, and the pooled estimate is drawn as a diamond at the bottom of a chart called a forest plot. Each horizontal line on the plot is one study. A vertical line marks no difference, and a diamond sitting clearly to one side of it points to benefit or harm.
The Handbook lists the advantages: better precision, the ability to answer questions no single study asked, and a way to settle conflicting claims. It also warns that meta-analyses have the potential to mislead seriously if study designs, biases within studies, variation between studies and reporting biases are not carefully considered.
What happens when the evidence is combined in time?
A 1992 paper in the New England Journal of Medicine by Joseph Lau and colleagues gives a striking answer. They ran a cumulative meta-analysis of 33 trials of intravenous streptokinase, a clot-dissolving drug for heart attacks, carried out between 1959 and 1988, adding each trial in turn. A consistent, statistically significant reduction in deaths was already clear by 1973, after just eight trials involving 2,432 patients.
The next 25 trials enrolled another 34,542 patients and barely changed the estimate. They only narrowed the confidence interval. Read alongside the Antman study, the message is uncomfortable: the evidence was there years before many experts acted on it, because nobody had gathered and combined it systematically.
What does the Cochrane logo show?
Cochrane's own logo is a forest plot. The story starts in 1972, when Liggins and Howie published the first randomised trial of the treatment in humans. According to Cochrane, the logo shows that corticosteroids given to women about to give birth prematurely can save the lives of their newborn babies. Several trials had shown a benefit, yet obstetricians were slow to adopt the treatment. A systematic review, first published by Crowley and colleagues and later updated, helped increase its use.
The 2017 Cochrane update by Roberts and colleagues included 30 studies with 7,774 women and 8,158 infants. Compared with placebo or no treatment, antenatal corticosteroids were associated with fewer perinatal deaths (risk ratio 0.72) and less respiratory distress syndrome (risk ratio 0.66), with the evidence for both rated as moderate quality. No single trial could have shown that as clearly.
Can you have a systematic review without a meta-analysis?
Yes. Sometimes the studies are too different in their participants, treatments or outcomes to combine sensibly, or there are too few, or they are too biased. Chapter 10 of the Cochrane Handbook opens with a section titled 'Do not start here!', warning that meta-analysis results can be very misleading if the question, eligibility criteria and data have not been handled carefully first.
The Handbook also discusses the old 'apples and oranges' objection: combining unlike studies can produce a meaningless average, although a broader question, such as the effect of a whole class of drugs, can sometimes justify pooling them. When pooling is not sensible, a good review compares the studies in words and tables and explains why it did not combine them.
Can a meta-analysis be wrong?
Yes. A meta-analysis inherits every flaw of the studies it combines, and it can be skewed if some studies, often those with disappointing results, never made it into print. In a 1997 New England Journal of Medicine study, Jacques LeLorier and colleagues compared 19 meta-analyses with 12 large randomised trials of 1,000 or more patients later published on the same questions. The earlier meta-analyses failed to predict the large trials' outcomes accurately 35% of the time. The picture was not all bad. The difference in the actual size of the effect was statistically significant in only 5 of the 40 outcomes compared. Most disagreements came down to one method finding a statistically significant effect while the other did not, which is a reminder that significance thresholds can exaggerate how far two results really differ.
Systematic review vs meta-analysis at a glance
- Systematic review: a method for finding, selecting and appraising all relevant studies on one question.
- Meta-analysis: a statistical method for combining the numerical results of two or more studies.
- A systematic review can stand alone when the studies are too different to pool.
- A meta-analysis without a systematic search behind it can cherry-pick studies, so its precise-looking result may mislead.
- Both are only as reliable as the studies they include.
When you see a headline saying 'a meta-analysis found', it is worth checking whether it came from a systematic review with a published protocol, how many studies and participants it included, whether the authors assessed risk of bias, and whether the individual studies broadly agreed with each other.
This article is for education about research methods only. It is not medical advice, so speak to a qualified health professional about any treatment decision.
Frequently asked
- What is the difference between a systematic review and a meta-analysis?
- A systematic review is a method for finding, selecting and appraising all the studies that answer one question, using rules set in advance to reduce bias. A meta-analysis is a statistical technique that combines the numerical results of two or more studies into one estimate. A meta-analysis is often one step inside a systematic review.
- Can you have a systematic review without a meta-analysis?
- Yes. If the studies are too different in their participants, treatments or outcomes, or too few or too biased, pooling them can produce a misleading average. The Cochrane Handbook warns that meta-analysis results can be very misleading without careful groundwork. A good review then compares the studies in words and tables and explains why it did not pool them.
- Is a meta-analysis always reliable?
- No. It inherits the flaws of the studies it combines and can be skewed if studies are missing. A 1997 New England Journal of Medicine study found that earlier meta-analyses failed to predict the results of 12 later large trials accurately 35% of the time. A precise-looking number is only as good as the evidence behind it.
- Why are systematic reviews considered strong evidence?
- Because they gather all the relevant trials using transparent rules, rather than relying on the studies an expert happens to know. A 1992 JAMA study found expert review articles on heart attack treatment often lagged behind the trial evidence, sometimes still recommending treatments that did not reduce deaths or were potentially harmful.
The studies and sources behind this page
- Cochrane. Chapter 1: Starting a review. In: Higgins JPT, Thomas J, Chandler J, et al, eds. Cochrane Handbook for Systematic Reviews of Interventions, version 6.5. Cochrane, 2024.
- Deeks JJ, Higgins JPT, Altman DG, McKenzie JE, Veroniki AA. Chapter 10: Analysing data and undertaking meta-analyses. In: Cochrane Handbook for Systematic Reviews of Interventions, version 6.5. Cochrane, 2024.
- Page MJ, McKenzie JE, Bossuyt PM, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 2021;372:n71.
- Antman EM, Lau J, Kupelnick B, Mosteller F, Chalmers TC. A comparison of results of meta-analyses of randomized control trials and recommendations of clinical experts. Treatments for myocardial infarction. JAMA 1992;268:240-8.
- Lau J, Antman EM, Jimenez-Silva J, et al. Cumulative meta-analysis of therapeutic trials for myocardial infarction. N Engl J Med 1992;327:248-54.
- Cochrane. Our name and logo.
- Roberts D, Brown J, Medley N, Dalziel SR. Antenatal corticosteroids for accelerating fetal lung maturation for women at risk of preterm birth. Cochrane Database Syst Rev 2017;3:CD004454.
- LeLorier J, Grégoire G, Benhaddad A, Lapierre J, Derderian F. Discrepancies between meta-analyses and subsequent large randomized, controlled trials. N Engl J Med 1997;337:536-42.
Lessons on this topic
Related reading
Educational content only. Not medical advice, diagnosis or treatment. Always consult a qualified healthcare professional before changing your health regimen.