Skip to Main Content
It looks like you're using Internet Explorer 11 or older. This website works best with modern browsers such as the latest versions of Chrome, Firefox, Safari, and Edge. If you continue with this browser, you may see unexpected results.

Systematic reviews

A guide to systematic reviews for health and social sciences

Construct a search

The search strategy for a systematic review needs to be as comprehensive as possible in order to capture all studies relevant to the review question. The strategy also needs to be transparent, rigorous and replicable as it is also published in your review. 

The general process in developing a search strategy is:

  1. Identify synonyms related to the key concepts in your research question (e.g. brainstorm using online dictionaries/resources)
  2. Identify the databases and other sources you need to search 
  3. Identify relevant database subject headings (e.g. MeSH - Medline, Emtree - Embase, etc) and apply to your search strategy
  4. Apply search techniques such as Boolean, Phrase Searching, Truncation and Wildcards
  5. Consider if certain concepts/keywords require to be searched in particular database fields e.g. Title/Abstract
  6. Consider if certain database filters need to be applied in order to narrow the search e.g. publication dates, age, sex, article type, language etc.
  7. Test, review and amend search as required

A useful starting point can be to check the search strategies in published systematic reviews for examples of how searches are structured and assist in keyword development. Your senior librarian can provide you with assistance in developing search strategies.

Synonyms (alternate words)

Brainstorming keywords for a search strategy for a systematic review is essential.  Use the concept table below to help structure your brainstorming process.

A comprehensive search would usually entail a combination of subject headings, plus a wide range of keywords/phrases for each concept. Depending on your topic in social sciences, you might find that keywords/phrases result in a better search than using subject headings.

NOTE! Not all databases will have subject heading searching and for those that do, the subject heading categories may differ between databases. This is because databases classify articles using different criteria.

Concept Table

This example is based on the research question:

What are rehabilitation interventions for older people following a stroke?

  1. Identify the main concepts from your research question, they are the headings for each column of your table;
  2. List your synonyms for each concept in the relevant column, including your overall concept;
  3. Use OR between synonyms, which will broaden the search for each concept (e.g stroke OR cerebrovascular accident OR CVA);
  4. Apply relevant search techniques such as truncation or phrase searching to each keyword;
  5. Each concept column will then be combined with AND. For boolean searching tips, review search smarter.
  6. Test your search in a database to ensure each concept appears in your search results. 

Example Medline search

The key fields to search are the:

Title OR abstract - use advanced search page to build a search with synonyms for an idea combined with OR

Subject Heading - use the database headings (eg MeSH or CINAHL headings) to find the subject term

  Concept 1 - Stroke AND Concept 2 - Older person AND Concept 3 - Rehabilitation
Title or Abstract stroke OR "cerebrovascular accident" OR "CVA"    elderly OR "old* person" OR aged OR senior*   "resistance train*" OR "physical activit*" OR exercis*
OR          
MeSH

STROKE

 

AGED

FRAIL ELDERLY

MIDDLE AGED

AGED 80, AND OVER

 

RESISTANCE TRAINING

EXERCISE

See below for a copy of a concept table to help you plan your search strategy.

Source: Queensland University of Technology systematic review library guide - Search the Literature

Bramer, W. M., de Jonge, G.,B., Rethlefsen, M. L., A.H.I.P., Mast, F., & Kleijnen, J. (2018). A systematic approach to searching: An efficient and complete method to develop literature searches. Journal of the Medical Library Association, 106(4), 531-541. doi:http://dx.doi.org/10.5195/jmla.2018.283

Suggested databases

Depending on your review question, you need to consider searching both multidisciplinary and subject-specific databases to ensure your search is comprehensive. Multidisciplinary databases such as Web of Science and Scopus contain a high volume of literature from a diverse range of subjects. They are helpful in locating literature on your topic from disciplines you might not have considered. On the other hand, subject-specific databases contain in-depth coverage of literature which is highly specific to a field. A balance of both databases is required in your search strategy.

For subject-specific databases that are relevant to your topic, ask your senior librarian for advice or for a comprehensive list of databases, check the Library's A-Z list of databases and refine by subject.

Databases which are commonly used for searching in health and social sciences include:

Databases that have controlled vocabulary/subject headings:
  • Medline
  • PubMed
  • CINAHL (nursing & allied health)
  • Embase
  • PsycInfo (EBSCOhost)
  • ERIC (EBSCOhost) (education)
  • ABI/Inform Complete (education, business, indigeneous)
  • Social Science database (Proquest)
Databases that search by keyword only:
  • Scopus (multidisciplinary)
  • Web of Science (multidisciplinary)
  • Informit Complete

Source: The University of Sydney library systematic review guide

Resources:

Translating your search across databases 

A quick guide from Flinders University Library 

and a comprehensive guide from ACU Library 

Grey literature and other sources

Searches for studies should be as extensive as possible in order to reduce the risk of publication bias and to identify as much relevant evidence as possible. Grey literature is unpublished material or has been distributed outside mainstream commercial publishing.

It may include reports, theses, government and non-government organisation publications, conference abstracts and proceedings, registries of clinical trials and prospective studies, and the results of hand searching or corresponding directly with authors.

You might even consider hand searching the contents pages of all issues of a select set of highly relevant journals. Searching grey literature is supported/mandated by the Cochrane Collaboration, the Campbell Collaboration, JBI and the Institute of Medicine (U.S.).

Seek assistance from your senior librarian if further questions.

Key Sources

These resources are a good starting point in your search for grey literature.

Library Catalogues

  • Trove which includes references and full-text links to Australian grey literature
  • Large libraries like the Australian National Library which often collect grey literature in paper form
  • WorldCat which holds millions of holdings from numerous libraries world-wide. WorldCat is a US-based, international network of libraries and services. Limit results of a search by selecting Dissertation/Thesis in the content box. Very occasionally there will be a link to an online abstract or copy of the thesis.

Repositories

Other Sources 

Cochrane advises:

  • searching within previous reviews on the same topic is highly desirable
  • checking reference lists in included studies and any relevant systematic reviews identified is mandatory. Searches for studies should be as extensive as possible in order to reduce the risk of publication bias and to identify as much relevant evidence as possible.

Evaluating Grey Literature

It is important to evaluate and appraise grey literature due to the variability of sources as some documents may not have been peer reviewed and the quality can vary. A possible checklist for evaluating grey literature is using AACODS (Authority, Accuracy, Coverage, Objectivity, Date and Significance).

  • Authority - Is the author/source credible? What are their qualifications? Do they have any expertise in the area?
  • Accuracy -  Is it supported by documented and authoritative references? Are the facts/figures, dates cited, and quality of evidence reliable and valid? Is there a clearly stated methodology? Is it in line with other work on the same topic
  • Coverage - Have limitations been imposed and are these stated clearly?
  • Objectivity - Can bias be detected? How are the claims justified? Is the purpose to promote a product/service?
  • Date - Can't find the date? Rule of the thumb is to avoid such material. How up to date is the information? Is the information cited and references included?
  • Significance - Is it relevant? Would it enrich or have an impact on your research?

Flinders University have developed a tool for evaluating grey literature, titled the AACODS Checklist (PDF, 560KB). This checklist can be downloaded to help you assess any types of grey literature you use.

Reporting your grey literature search

Reporting on a grey literature search is not as straightforward as for a bibliographic database. Where possible you should aim to record the following information:

  • Name of web-site or resource used
  • URL (if appropriate)
  • Date of search
  • Keywords used for searching or details of how you browsed

Resources:

Bonato, S. (2018). Searching the grey literature : a handbook for finding annual reports, working papers, white papers, government documents, and more. Rowman & Littlefield Publishers.

Stansfield, C., Dickson, K., Bangpan, M. (2016). Exploring issues in the conduct of website searching and other online sources for systematic reviews: how can we be systematic? Systematic Reviews, 5(1):191. https://doi.org/10.1186/s13643-016-0371-9

CADTH. (2019). Grey Matters: a practical tool for searching health-related grey literature.

CADTH. (2013). Finding the Evidence: Literature Searching Tools in Support of Systematic Reviews.

Canberra University has an extensive Library Guide to Grey Literature in health.

Flinders University has a useful guide to Googling for grey literature

When to stop searching

There comes a point where the rewards of further searching may not be worth the the effort needed to capture additional references. The call to abort further searching depends on the question a review addresses and the resources that are available. Check the resources below for more information:

When to stop searching 4.4.11 The Cochrane handbook

The PRESS Guideline provides a set of recommendations and a checklist which can be used to evaluate electronic search strategies.

Peer review of electronic search strategies (PRESS)

The full guideline statement and checklist document may be accessed via the following open access article from the Journal of Clinical Epidemiology:

​McGowan, J., Sampson, M., Salzwedel, D.M., Cogo, E., Foerster, V, Lefebvre C. (2016). PRESS Peer Review of Electronic Search Strategies: 2015 guideline statement. J Clin Epidemiol. 2016 Jul;75:40-6. https://doi.org/10.1016/j.jclinepi.2016.01.021

When conducting a systematic review, your aim is to search enough databases to be able to state with confidence that the literature has been comprehensively searched. Indicators of a comprehensive search strategy include:

  • you look in a different database and don’t retrieve any new papers
  • you can identify no new data (reach saturation)
  • you use a rule to decide when to stop searching

Source: The University of Sydney Library Systematic review guide 

Search filters

Subject area filters

A search filter is a defined search strategy designed to find certain types of articles in a particular database. These search filters have been tested by experts and are a quick method of applying a search in the areas below.

CareSearch Australia: find evidence-based literature via PubMed for palliative care and related topics

Integrated care: search integrated care literature via PubMed

Aboriginal and Torres Strait Islander Health: search for literature about ATSI populations and health issues via PubMed

Search Blocks (Biomedische Informatie): add 'blocks' to your database searches to limit them by topic