If you are interested in doing a systematic review, check out the Library's systematic review services for faculty and consider signing up for the So You Want To Do A Systematic Review Workshop.
Grant, M. J., & Booth, A. (2009). A typology of reviews: An analysis of 14 review types and associated methodologies. Health Information and Libraries Journal, 26(2), 91-108. doi:10.1111/j.1471-1842.2009.00848.x
A systematic review begins with a clearly defined question accompanied by established inclusion and exclusion criteria.
Once you have an initial idea, search for already published literature on the topic. If there is already a systematic review article, then another review on the same topic is not needed. Additionally, reviewing published systematic reviews can help you to better frame your own research question.
If you find a research article that matches your topic, save it. Articles that exemplify your research topic are called "gold-standard" articles, and are used as examples during the search process (Step 3).
Databases for Finding Already Published Reviews
What is a systematic review protocol?
Based on Cochrane, it is a pre-defined plan and the proposed approach for a systematic review. It outlines the question that the review authors are addressing, detailing the criteria against which studies will be assessed for inclusion in the review, and describing how the authors will manage the review process. The protocol also serves as a notification of your plans to other researchers, so that no one will attempt the same project. It's strongly suggested to register your systematic review protocol once it's written. The purpose of registering a systematic review is to reduce publication bias, enhance transparency, and avoid duplication of effort.
A good way to familiarize yourself with writing systematic review research protocols is to take a look at those registered on PROSPERO- the International Prospective Register of Systematic Reviews. Once you have written a draft of your protocol, have it peer reviewed by someone outside of your research team. If it is registered in PROSPERO, it will also be open for peer review by other researchers.
Key information for writing a systematic review protocol include:
Besides PROSPERO, there are other organizations that you might search for a registered systematic review protocol:
Key messages
Search/Methodology Filters
Methodology filters can help get rid of undesirable study designs so as to reduce the size of a large retrieval. However systematic review attempts to maximize sensitivity in terms of retrieving all relevant documents. Therefore it is always preferable not to employ filters.
Sometimes retrieval sets are literally unmanageable. You‘d probably want to implement a methodology filter. If so, use at all possible validated filters, which have been tested against gold-standard sets of bibliographic records.
After the search is completed, the research team will review the abstracts based on specific inclusion/exclusion criteria independently. Once all the abstracts have been reviewed by each team member, the research team should meet to resolve any conflicts.
Once any conflicts from the abstract review are resolved, the abstracts voted for inclusion will move to the full-text review. The research team should gather the full-text articles and follow the same independent review process with inclusion/exclusion criteria.
You might consider using systematic review software or other data management tools to assist in this step and the full-text review step.
Data extraction and analysis can be done in a few different ways. You could develop a data extraction form that the research team will use. You could also use an existing software or tool to complete this step.
Data Extraction Slides from AHRQ
The slides describe why data extract is important, data extraction challenges, forms, methods to collect data accurately and effectively, etc.
You can learn more about data extraction and management here.
At the end, it is important to disseminate the information gathered by the research team. The format of the written report might depend on your targeted journal(s) for publication. Refer to their author and submission guidelines.
Generally, the written report includes summaries and descriptions for each of the steps.
Examples of systematic review articles:
Buchwald, H., Avidor, Y., Braunwald, E., Jensen, M. D., Pories, W., Fahrbach, K., & Schoelles, K. (2004). Bariatric surgery: A systematic review and meta-analysis. JAMA, 292(14), 1724-1737. doi:10.1001/jama.292.14.1724. PMID: 15479938
Chaudhry, B., Wang, J., Wu, S., Maglione, M., Mojica, W., Roth, E., . . . Shekelle, P. G. (2006). Systematic review: Impact of health information technology on quality, efficiency, and costs of medical care. Annals of Internal Medicine, 144(10), 742-752. PMID: 16702590
Due to the methodology and standards for systematic reviews, this type of research is a significant time commitment.
The Cochrane Handbook for Systematic Reviews of Interventions estimates a 12-month timeframe for completion.
| Month | Activity |
| 1-2 | Preparation of Protocol |
| 3-8 | Search for Published and Unpublished Studies |
| 2-3 | Pilot Test of Eligibility Criteria |
| 3-8 | Inclusion Assessments |
| 3 | Pilot Test of 'Risk of Bias' Assessment |
| 3-10 | Validity Assessments |
| 3 | Pilot Test of Data Collection |
| 3-10 | Data Collection |
| 3-10 | Data Entry |
| 5-11 | Follow-up of Missing Information |
| 8-10 | Analysis |
| 1-11 | Preparation of Written Report |
| 12- | Keeping the Review Up-to-date |
Adapted from Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2011. Available from https://training.cochrane.org/handbook
General Biomedical Databases
Subject Specific Databases
Databases for Finding Reviews
Grey Literature
Grey literature refers to "information produced on all levels of government, academics, business and industry in electronic and print formats not controlled by commercial publishing" ie. where publishing is not the primary activity of the producing body." — ICGL Luxembourg definition, 1997. Expanded in New York, 2004. They include conference abstracts or papers, hard to find studies, reports, or dissertations, governmental or private sector research, ongoing or unpublished clinical trials, etc.
| Type | Aim | Question Format | Question Example |
| Effectiveness | To evaluate the effectiveness of a certain treatment/practice in terms of its impact on outcomes | Population, Intervention, Comparator/s, Outcomes (PICO) |
What is the effectiveness of exercise for treating depression in adults compared to no treatment or a comparison treatment? |
| Experiential | To investigate the experience or meaningfulness of a particular phenomenon | Population, Phenomena of Interest, Context (PICo) | What is the experience of undergoing high technology medical imaging (such as Magnetic Resonance Imaging) in adult patients in high income countries? |
| Costs/Economic Evaluation | To determine the costs associated with a particular approach/treatment strategy, particularly in terms of cost effectiveness or benefit | Population, Intervention, Comparator/s, Outcomes, Context (PICOC) |
What is the cost effectiveness of self-monitoring of blood glucose in type 2 diabetes mellitus in high income countries? |
| Prevalence and/or Incidence | To determine the prevalence and/or incidence of a certain condition | Condition, Context, Population (CoCoPop) |
What is the prevalence/incidence of claustrophobia and claustrophobic reactions in adult patients undergoing MRI? |
| Diagnostic Test Accuracy | To determine how well a diagnostic test works in terms of its sensitivity and specificity for a particular diagnosis | Population, Index Test, Reference Test, Diagnosis of Interest (PIRD) |
What is the diagnostic test accuracy of nutritional tools (such as the Malnutrition Screening Tool) compared to the Patient Generated Subjective Global Assessment amongst patients with colorectal cancer to identify undernutrition? |
| Etiology and/or Risk | To determine the association between particular exposures/risk factors and outcomes | Population, Exposure, Outcome (PEO) |
Are adults exposed to radon at risk for developing lung cancer? |
| Expert opinion/policy | To review and synthesize current expert opinion, text or policy on a certain phenomena | Population, Intervention or Phenomena of Interest, Context (PICo) |
What are the policy strategies to reduce maternal mortality in pregnant and birthing women in Cambodia, Thailand, Malaysia and Sri Lanka? |
| Psychometric | To evaluate the psychometric properties of a certain test, normally to determine how the reliability and validity of a particular test or assessment | Construct of interest or the name of the measurement instrument(s), Population, Type of measurement instrument, Measurement properties |
What is the reliability, validity, responsiveness and interpretability of methods (manual muscle testing, isokinetic dynamometry, hand held dynamometry) to assess muscle strength in adults? |
| Prognostic | To determine the overall prognosis for a condition, the link between specific prognostic factors and an outcome and/or prognostic/prediction models and prognostic tests | Population, Prognostic Factors (or models of interest), Outcome (PFO) |
In adults with low back pain, what is the association between individual recovery expectations and disability outcomes? |
| Methodology | To examine and investigate current research methods and potentially their impact on research quality | Types of Studies, Types of Data, Types of Methods, Outcomes (SDMO) | What is the effect of masked (blind) peer review for quantitative studies in terms of the study quality as reported in published reports? |
Munn, Z., et al. (2018). "What kind of systematic review should I conduct? A proposed typology and guidance for systematic reviewers in the medical and health sciences." BMC Medical Research Methodology 18(1): 5. doi: 10.1186/s12874-017-0468-4