Systematic reviews

After you've conducted your searches and exported all your results into a reference manager, you can start screening articles. During this step, reviewers select which studies are relevant based on their inclusion and exclusion criteria.

To control bias and minimise the risk of excluding relevant studies, screening is usually done by two or more independent reviewers. They talk through any disagreements in decision making over study inclusion or exclusion.

A standard workflow for this stage might be:

1. Remove duplicate records from your pool of results
2. Examine titles and abstracts to remove articles which obviously don't meet the inclusion criteria
3. Find the full text of all potentially relevant articles
4. Link together any reports which discuss the same study
5. Screen full text articles and assess for inclusion in the review against the criteria outlined in your protocol
6. Make any final decisions on study inclusion

The PRISMA flow diagram visually summarises the screening process. It makes the selection process transparent and clear to the reader by reporting on decisions made at various stages of the systematic review. When you're excluding articles at the full-text stage, you need to specify the reasons for exclusion. It's important to document this thoroughly.

Pre-screening:   Record the numbers of results from each database or source recorded before screening commences 

Deduplication: Remove duplicated articles from the combined result set.

Title/abstract screening:  Each reviewer will need to read titles and abstracts to see if they match the inclusion/exclusion criteria or have some value to the systematic review.  This is done separately by each reviewer to ensure minimal bias.  Results are then compared.

Full-text screening:  Included articles are located in full-text prior to multiple reviewers individually read through the full-text of included articles to fine-tune the final collection of articles that will contribute to the review.

Source: The University of Sydney Systematic review library guide 

When reviewing the final search results from your chosen databases (and other sources if relevant), multiple reviewers (you and your supervisors or co-reviewers) will make decisions on which articles to include and exclude based on the criteria specified in your protocol. The first stage of this is usually based on titles and abstracts, then a full text analysis follows before data extraction.

EndNote:

Use EndNote to store and screen references. EndNote is useful for the initial title/abstract screen and sourcing articles required for the full-text screening process.

Rayyan: 

Rayyan is a freely available program that helps screen studies for a systematic review by automatically identifying disagreements and allowing the use of keywords to help identify relevant studies. Rayyan also has a mobile app for iPhone, iPad and Android devices. 

Covidence:

Systematic Review management software. Subscription required, you can sign up for a trial review of under 500 articles for free or to purchase a yearly licence. 

Abstrackr:

A web-based tool for collaborative screening.

Critical appraisal involves checking the quality, reliability and relevance of the studies in the review in relation to the research question. It appraises each study in terms of the following aspects:

1. Does this study address a clearly focused question?
2. Did the study use valid methods to address this question?
3. Are the valid results of this study important?
4. Are these valid, important results applicable to my patient or population?

Where to start:

1. Look for a research problem or a problem statement (usually in the abstract, title or introduction)
2. What is the purpose? Check the abstract
3. Is it a literature review?
4. Is it a hypothesis or research question?
5. What is the sample or population of interest?
6. What is the type of research or study design?

Consider looking through the How to read a paper series by Trisha Greenhalgh, which explain how to interpret different kinds of research papers.

There are various critical appraisal tools available to assist you with this process:

• The Centre for Evidence- Based Medicine (CEBM) has a number of worksheets to guide you.
• Cochrane risk-of-bias tool for randomized trials (RoB 2), the recommended tool for use in Cochrane Reviews.
• The Critical Appraisal Skills Programme (CASP) This set of eight critical appraisal tools are designed to be used when reading research, these include tools for Systematic Reviews, Randomised Controlled Trials, Cohort Studies, Case Control Studies, Economic Evaluations, Diagnostic Studies, Qualitative studies and Clinical Prediction Rule.
• Joanna Briggs Institute (JBI) offers critical appraisal checklists for various study types. The tools are available for download as a PDF.
• NHMRC List of Good Practice Tools to Assess Risk of Bias

Source: Latrobe University Library Critical appraisal for health