What is a systematic review?
This library guide provides an overview of what a systematic review is and how librarians can help students and researchers at the Australian Catholic University during a systematic review process. This guide focuses on systematic reviews in the health sciences and social sciences disciplines.
A systematic review is a review of a clearly formulated question that uses systematic and reproducible methods to identify, select and critically appraise all relevant research, and to collect and analyse data from the studies that are included in the review. A systematic review can be either quantitative or qualitative or both.
Systematic reviews are an approach used in health, education and social policy (as part of evidence based policy or practice). It's much more than a literature review; it follows a strict methodology which means it's transparent, rigorous and replicable.
A quantitative systematic review will include studies that have numerical data. A qualitative systematic review derives data from observation, interviews, or verbal interactions and focuses on the meanings and interpretations of the participants. It will include focus groups, interviews, observations and diaries.
Before you start:
Ask yourself - is your research topic best suited for a systematic review? You might find that a systematic review isn't the best study design for your research topic, due to the scope of your question or the body of literature available.
- Review the Cochrane video below
- What type of review? Check the review decision tree (Cornell University) or consult Right Review
- Read some examples of different reviews for health sciences and social sciences.
Table of differences between reviews
| Literature Review | Scoping Review | Systematic Review | |
|---|---|---|---|
| Description | Describes, summarises and evaluates existing information. | Preliminary assessment of potential size and scope of available literature. | Seeks to systematically search for, appraise and synthesise research evidence. |
| Question | An overview not necessarily focused on a single question. | Addresses a broader research question or set of questions. | Focused on a single question. |
| Protocol | No protocol is included. | Ideally, a protocol or plan is included. | Protocol is planned and specific. |
| Objectives | The goal of the review may be to place one's own research within the existing body of knowledge, or to gather information that supports a particular viewpoint. | Often conducted in preparation for conducting a systematic review. | Starts with a well-defined research question to be answered by the review. Reviews are conducted with the aim of finding all exisiting evidence in an unbiases, transparent and reproducible way. |
| Criteria | Criteria is not specified. | Eligibility criteria are clearly defined before the review is conducted but may be revisited throughout the review. | Inclusion & exclusion criteria is stated before the review is started. |
| Search Strategy | Searches are not exhaustive or fully comprehensive. |
Comprehensive sources and explicit and reproducible search strategy. Completeness determined by time/scope restraints. |
Comprehensive search conducted in a systematic way that can be repeated. The process is well-documented and reported. |
| Study selection | Often lacks clear reasons for why studies were included or excluded from review. | Often broad - may differ depending on resources found. | Reasons for including and excluding studies are explicit and informed by the research question. |
| Quality appraisal | Often do not consider study quality or potential biases in study design. | Variable - typically not done, or done in narrative form. | Rigorous critical appraisal, and evaluation of study quality. |
| Recommended timeframe | Weeks to months. | 2-6 months. |
9-12+ months. |
Adapted from: University of South Australia Library Guide to Systematic Review and Boston University Library Guide to Systematic Reviews in Social Sciences
Steps involved in a systematic review
- Construct question
- Scoping search
- Devise protocol (plan)
- Conduct comprehensive search
- Select studies against eligibility criteria
- Appraise studies using quality checklist
- Data extraction
- Analyse results
- Interpret findings
- Disseminate report
A systematic review requires:
Time
A systematic review is a rigorous, time-consuming undertaking.
A team
You need to work with subject experts; at least one other reviewer for screening citations for relevance, and a librarian who can advise on suitable resources and develop the search strategies, if required. Someone is also required to manage the data and write up the report.
An initial scoping search
This will reveal if a systematic review has already been undertaken on your topic as well as highlight gaps in the knowledge base. A scoping search will also allow you to develop and pilot inclusion/exclusion criteria, and should give you an initial sense of the quantity of existing primary research on your topic (and therefore the scale of the project).
A clearly focused question
This should flow from your initial search. The question should be worthy of an answer and should strike a suitable balance between being too broad or too narrow in scope. If appropriate, use the PICO framework to identify the important concepts in your question.
Clear inclusion/exclusion criteria
These should be based on your question. Consider all aspects of the topic such as age groups, geographic regions, types of study designs, languages, and the outcome measures that need to be described. Clear eligibility criteria will make it easier to identify relevant articles at the screening stage and prevent you being distracted by interesting but irrelevant studies.
A comprehensive literature search
A systematic approach to searching the literature for Systematic Reviews will help to identify all of the best available literature that addresses your specific question. This must include a search for unpublished studies as well as published ones. It should target an exhaustive set of relevant databases and include a search for grey literature or other relevant sources if appropriate. Search strategies need to be well documented, including enough detail that they can be replicated. The numbers of results from each database search should also be included.
Critical appraisal
Each study meeting the inclusion criteria must be appraised for quality (internal validity) as well as applicability (external validity or generalisability). A critical appraisal tool such as the CASP checklist or the Joanna Briggs Institute (JBI) tools can guide you in this.
Good citation management
EndNote is a useful tool for organising your search results. After exporting your results from multiple database searches and other sources into a new library, within separate groups, you can then deduplicate for a complete set of results ready for the screening process. For any assistance with EndNote, please contact your librarian.
Source: Flinders University Library Guide to Systematic Reviews
Examples of reviews
Most systematic reviews are currently conducted in fields like health and medicine. Increasingly, systematic reviews are being published in other disciplines such as humanities and education. These articles provide examples of different types of reviews:
Review Types:
Literature (Narrative) Review
- Hindle, A., de la Piedad Garcia, X., & Brennan, L. (2017). Early post‐operative psychosocial and weight predictors of later outcome in bariatric surgery: a systematic literature review. Obesity Reviews, 18(3), 317-334. https://go.openathens.net/redirector/acu.edu.au?url=https%3A%2F%2Fdoi.org%2F10.1111%2Fobr.12496
- Owusu-Addo, E., Ofori-Asenso, R., Batchelor, F., Mahtani, K., & Brijnath, B. (2021). Effective implementation approaches for healthy ageing interventions for older people: A rapid review. Archives of gerontology and geriatrics, 92, 104263. https://doi.org/10.1016/j.archger.2020.104263
Scoping Review
- Riegel, B., Westland, H., Iovino, P., Barelds, I., Slot, J. B., Stawnychy, M. A., … & Jaarsma, T. (2021). Characteristics of self-care interventions for patients with a chronic condition: A scoping review. International Journal of Nursing Studies, 116, 103713.
https://go.openathens.net/redirector/acu.edu.au?url=https%3A%2F%2Fdoi.org%2F10.1016%2Fj.ijnurstu.2020.103713
Systematic Review
-
Morris, H., Skouteris, H., Edwards, S., & Rutherford, L. (2015). Obesity prevention interventions in early childhood education and care settings with parental involvement: a systematic review. Early Child Development and Care, 185(8), 1283–1313. https://go.openathens.net/redirector/acu.edu.au?url=https%3A%2F%2Fdoi.org%2F10.1080%2F03004430.2014.991723
-
van Rensburg, D. C. C. J., van Rensburg, A. J., Fowler, P., Fullagar, H., Stevens, D., Halson, S., … & Roach, G. D. (2020). How to manage travel fatigue and jet lag in athletes? A systematic review of interventions. British Journal of Sports Medicine. 54:960-968 http://dx.doi.org/10.1136/bjsports-2019-101635
Systematic Review Protocol
- Dol, J., Richardson, B., Murphy, G. T., Aston, M., McMillan, D., & Campbell-Yeo, M. (2019). Impact of mHealth interventions during the perinatal period on maternal psychosocial outcomes: a systematic review protocol. JBI Database of Systematic Reviews and Implementation Reports, 17(12), 2491-2498. https://go.openathens.net/redirector/acu.edu.au?url=https%3A%2F%2Fdoi.org%2F10.11124%2FJBISRIR-D-19-00003
For an overview of all types of reviews, the articles below are very useful.
Grant, M.J. and Booth, A. (2009), A typology of reviews: an analysis of 14 review types and associated methodologies. Health Information & Libraries Journal, 26: 91-108. https://doi.org/10.1111/j.1471-1842.2009.00848.x
Munn, Z., Peters, M.D.J., Stern, C., Tufanaru, C., McArthur, A., & Aromataris, E. (2018). Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Medical Research Methodology,18, 143. https://doi.org/10.1186/s12874-018-0611-x
Sutton, A., Clowes, M., Preston, L. & Booth, A. (2019). Meeting the review family: exploring review types and associated information retrieval requirements. Health Information and Libraries Journal, 36, 202-222. https://doi.org/10.1111/hir.12276
Help
Carrying out a systematic review is a long and elaborate process. The structure and topic of a systematic review must be devised by the researcher and supervisor, or research team, as a first step.
For assistance with systematic reviews, book a consultation with your librarian.
Last reviewed: December 2023