Writing a systematic review: Part 1

Authors: Shania Liu, Deanna Mill, Kenneth Lee and Amy Page | Members of the 2022 SHPA Research Leadership Committee

[First published in Pharmacy GRIT Vol 5; Summer 2021–2022. Some URLs and references have been updated]

Advanced Pharmacy Australia (AdPha) proudly supports the Research Toolkit series, which aims which aims to support members in conducting and publishing their research. This series is coordinated by the AdPha Research Leadership Committee, and hence shares the insights and experience of our most research-passionate members. If you’re keen to make a difference to patient care, not just in your daily practice, but in improving practice itself, then this series is for you.

Contents: 

What is a literature review and why conduct one?
Types of literature review: How are you capturing your data?
Developing your research question
Types of data to include
Types of articles to include
Sourcing relevant articles
Methods to organise your data
What next?

Useful resources
References

Introduction

If you may have a clinical question, you know that reviews provide a good summary of the evidence, but why are some called systematic reviews? Why would you write one? And how do you write one?

This series aims to break down the process of writing a systematic review while providing tips and references to handy tools along the way. If you’re new to research writing, or would like some general tips, see the first entry in the Research Toolkit Series: 'Writing your first research paper: A practical guide for clinicians'. Before we get into the details of writing a systematic review, it is important that we understand literature reviews more generally. Let’s take a step back and look at an introduction into the world of literature reviews.

What is a literature review and why conduct one?

A literature review, like its name suggests, involves the development of a summary of the available research on a specific topic.

Literature reviews are done by compiling a number of articles on a certain topic, like collecting individual pieces of a jig-saw puzzle. In the process of conducting a literature review, we put these pieces together to see the overall picture. This helps us to draw evidence-based conclusions with a better understanding of the overall research on a topic.

Types of literature reviews: How are you capturing your data?

There are several types of literature reviews, each involving different methods and serving a different purpose. Take a look at the name of each literature review type below for a clue about what it involves. Before starting your literature review, it’s a good idea to pick the one which best fits your timeframe and needs.

Narrative reviews

The aim of a narrative review is to tell a story about a particular topic. These reviews can provide a brief or in depth overview of a topic. However, the approach to finding relevant research may be unstructured and articles may be cherry-picked (e.g. only the most recent or topical publications). Narrative reviews don’t require a predetermined research question or defined plan to search databases, generally making them quicker to conduct.

Systematic reviews

A systematic review involves a very thorough comb-through of the research to find all the available evidence on a defined research question. This involves a pre-determined plan of how you’re going to source relevant articles, called a search strategy. Usually at least two people independently screen the available literature and analyse the information from relevant articles. This information is then synthesised to formulate an answer to a focused research question (more on this later).

A systematic review also differs from other types of reviews because it involves a rigorous assessment of the quality of the included studies (more on this later too). You should clearly record the way you perform your systematic review so that another researcher can perform the same review using your methods (this is called reproducible research). As this is a rigorous process, systematic reviews can take months, or even years, to complete. While it’s hard work, a systematic review is considered the ‘gold standard’ literature review as it provides an unbiased, balanced summary of a topic. Since systematic reviews are quite rigorous, researchers often prepare and publish the protocol of their search methods before doing the search. These protocols are published using online international registers such as PROSPERO or Open Science Framework.

Scoping reviews

A scoping review aims to assess the potential scope, or how much available research there is on a topic. These reviews are carried out using a methodical process similar to a systematic review and involve a search strategy. This type of review is helpful for answering broad, background questions such as ‘What information on this topic is available in the literature?’ and identifying gaps in research. As such, they tend to incorporate both published and unpublished research (called grey literature).

Scoping reviews also provide an unbiased, balanced summary of a topic, however the research question is often more open-ended, compared with systematic reviews which focus on specific study end-points. Sometimes the outputs of the scoping review are checked (through consultation) with those working in the field before it is published, to provide a real-world perspective of the findings. Like systematic reviews, the protocol of a scoping review can also be published online before researchers start searching the literature.

PRO TIP FOR READERS:

  • Take a look at the reporting guideline for systematic reviews – the Preferred Reporting Items for Systematic Reviews (PRISMA)1,which can be used as a checklist for content to include in your review.
  • For scoping reviews, the PRISMA extension for scoping reviews (PRISMA-ScR)2 is available.

Rapid reviews

As health professionals, we know that sometimes we need health information quickly and a systematic review will take too long. A rapid review involves a methodical search of the literature on a topic of interest but is done in a shorter time frame by taking a few shortcuts, e.g. we may reduce the number of research databases that are searched, limit the inclusion criteria by article release date, or only have one person screen articles. As a result, rapid reviews provide answers to a research question in a more timely manner but may be prone to some bias, such as selection bias, which may happen when just one researcher decides what articles are included in the review. These are the reviews health professionals may conduct to answer health questions in a quick time span.

Umbrella reviews

No, we’re not looking for the best rated umbrellas here. In a research context, an umbrella review is a literature review of systematic reviews,’ sometimes called an overview of reviews’. These are undertaken if there are numerous systematic reviews on a certain topic and aim to provide an overarching conclusion by combining the key messages from each of these reviews.

Developing your research question

Now that you’ve landed on the type of review which best suits your topic, let’s dive deeper to look at the core of your literature review (and indeed any research article): your research question. There are two main types of research questions: background questions and foreground questions.

Background questions ask for general knowledge about a topic. These are usually broad questions looking at background information on a topic, e.g. ‘What causes insomnia?’.

Foreground questions ask for specific knowledge about a certain topic. This topic of interest is more strictly defined and involves 3–4 essential components. As a result, foreground questions are preferred for informing clinical decisions or answering clinical questions, e.g. ‘What is the effectiveness of rivaroxaban compared with warfarin on the incidence of stroke among adults with atrial fibrillation?’.

There are some handy mnemonics out there to help you define the key concepts of your research question.

Mnemonics

A mnemonic often used for systematic reviews is PICO: Population/Problem, Intervention, Comparison, Outcomes. Let’s break down what each letter of PICO means with a working example in Table 1.

If we put our examples from above together, the PICO question becomes ‘Compared with usual practice, does education provided to prescribers reduce the quantity of benzodiazepines prescribed for nursing home residents?’

Table 1. Example of the Population/Problem, Intervention, Comparison, Outcomes mnemonic.
PICO Element Meaning Example
Population/Problem What group of individuals / problem are we interested in? Nursing home residents
Intervention What is the main intervention / exposure we’re measuring the effect of? Prescriber education
Comparison Is there an alternative to compare with the intervention? Note: there doesn’t always have to be a comparison. No prescriber education (usual practice)
Outcomes What is the endpoint that I’m measuring? Quantity of benzodiazepines prescribed

For scoping reviews, we have a slightly simpler mnemonic, PCC: Population/Problem, Concept, Context. Let’s break down what PCC means and work through an example, in Table 2.

Putting our PCC question above, our example question becomes ‘What non-pharmacological treatments are used to reduce migraines among adults?’

Table 2. Example of the Population/Problem, Concept, Context mnemonic.
PCC Element Meaning Example
Population/Problem What group of individuals or problem are we interested in? Migraines
Concept What is the core concept of your review? This can include the interventions or outcomes you’re interested in. Non-pharmacological treatment options
Context What’s the setting you’re interested in? This can include different ethnic groups, demographics or geographical locations. Adults

PRO TIP FOR READERS:

  • Your research question doesn’t necessarily have to follow the order of PICO or PCC – think of these more as checklists to make sure your question is comprehensive.

Types of data to include

Something else to consider is the type of information or data you’d like to include in your literature review. The names of the types of data to include in your literature review offer clues about what they involve.

Quantitative data

Quantitative data has its name derived from ‘quantity’ and involves numeric information, e.g. ‘pain intensity on a scale of 1 to 10’.  

Qualitative data

Qualitative data looks at different ‘qualities’ about what you’re measuring by using words to describe information, e.g. ‘What are patients’ experiences of pain?’.

Mixed-methods

As its name implies, collecting mixed-methods data involves looking at both quantitative and qualitative data.  

Types of articles to include

There are many different types of articles which you can consider including in your review. The two most common article types are original research articles and literature reviews. These are usually published in peer-reviewed research journals, where ‘peer-reviewed’ refers to a process where articles submitted to the research journal are reviewed by your ‘peers’, usually an independent person with an understanding of your article topic, who provide feedback to improve the quality of your work. This process helps ensure articles published in such journals are of a standard which is acceptable to experts in the area.

Editorials, conference abstracts and opinion pieces are not usually included in systematic reviews as these may not report on original research or undergo the peer-review process.

There are also articles that haven’t been published in a research journal, sometimes called ‘grey literature’. These can include government reports, policy statements, unpublished conference proceedings, research theses or dissertations, professional (rather than research) journals like Pharmacy GRIT and newsletters. It can sometimes be tricky to find relevant information in the grey literature because they’re not organised in research databases like published studies are, but there are search engines that provide access to certain types of grey literature, e.g. ProQuest Dissertations and Theses Global.

Sourcing relevant articles

After determining your research question and the types of articles you plan to include in your literature review, the next step is to plan how you’ll source the articles you’re interested in. This is called a ‘search strategy’ and involves specifying the research databases you plan to use as well as the search terms you’ll apply to these databases. Some research databases use special language called subject headings and syntax to find articles. We recommend partnering with your hospital or university librarian to help translate your research question into a search strategy. Stay tuned for more information on formulating your search strategy as we will cover this topic in more depth in coming issues.

Methods to organise your data

Once you’ve found relevant articles for your literature review, it’s time to organise the information to help answer your research question. It’s a good idea to take a look at the quality of the research articles you’ve included in your review by undertaking a ‘quality assessment’ which involves checking your included articles for potential sources of bias. Bias can be happen in research studies when an error is introduced at some stage, resulting in skewed findings away from the ‘true’ result3 and may impact the reliability or accuracy of a study’s findings. There are a number of quality assessment tools you can use to guide you; the tool you select will depend on the type of studies included in your literature review.4

PRO TIP FOR READERS:

Examples of quality assessment tools include:

There are a number of ways you can combine your data:

Qualitative synthesis

Sometimes the articles we want to include in our review may vary so much that it seems like we’re comparing apples to oranges. A simple way to compare these articles is to describe what they did and found. This is called qualitative (using words to describe information) synthesis (combining information). 

Meta-analysis

Say you’ve done a systematic review on ‘the effect of paracetamol on body temperature among children in the community with a fever above 38°C’ and you find several studies reporting a reduction in body temperature. How would you determine the overall effect of paracetamol on body temperature from these studies? A meta-analysis is a statistical method for combining quantitative data from multiple studies to look at the overall effect. In order for a meta-analysis to be possible though, the treatment and variable you’re measuring need to be consistent between studies.

PRO TIP FOR READERS:

  • For more information on conducting a systematic review and/or meta-analysis, refer to Cochrane’s Guides and Handbooks.
  • A handy free software to help you perform a meta-analysis is Review Manager (RevMan), also by the Cochrane Collaboration.

Network meta-analysis

In clinical practice, there are countless situations where you need to decide the best treatment option for your patient. However, research on head-to-head comparisons (where we compare one drug to another drug) are often limited to a small handful of combinations. A network meta-analysis is a technique that allows us to estimate the relative effects of two (or more) interventions that have not been compared directly within individual trials.

Figure 1. Using a network analysis to compare multiple interventions.

As shown in Figure 1, derived from the Cochrane Handbook5, we can use a network analysis to compare multiple interventions. The figure shows how meta-analyses of randomised controlled trials can be conducted to obtain direct estimates of intervention effects comparing A vs. B and A vs. C. Although we don’t have a study comparing the effect of interventions B and C directly, we can use a network meta-analysis to provide an indirect estimate of intervention effects comparing B and C.

For another example, take a look at a network analysis by Cipriani et al which compares the efficacy and acceptability of 21 antidepressants for the treatment of major depressive among adults.6

Meta-synthesis

A meta-synthesis is a statistical method for interpreting qualitative data from multiple studies. Rather than combining data to look at an overall numerical effect, as done in meta-analysis, meta-synthesis instead aims to bring together the findings of qualitative studies. By interpreting these findings as a group, new meanings can be identified, allowing us to broaden our understanding and generate new interpretations about a particular concept.7

What next?

After deciding on your literature review type, research question, and the data you’ll include, you’re ready to proceed to the next step of writing a literature review: preparing a protocol on how you’ll search the literature, including a search strategy.

We hope this overview of literature reviews is helpful. Stay tuned for more in-depth information on writing systematic reviews in future issues of Pharmacy GRIT.

Useful resources

References

1. Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 2021; 372: n71.

2. Tricco AC, Lillie E, Zarin W, O'Brien KK, Colquhoun H, Levac D, et al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med 2018; 169: 467–73.

3. Delgado-Rodriguez M, Llorca J. Bias. J Epidemiol Community Health 2004; 58: 635–41.

4. Ma L-L, Wang Y-Y, Yang Z-H, Huang D, Weng H, Zeng X-T. Methodological quality (risk of bias) assessment tools for primary and secondary medical studies: what are they and which is better? Mil Med Res 2020; 7: 1–11.

5. Chaimani A, Caldwell DM, Li T, Higgins JPT, Salanti G. Chapter 11: Undertaking network meta-analyses. In: Higgins J, Thomas J, Chandler J, Cumpston M, Li T, Page M, Welch V, editors. Cochrane Handbook for Systematic Reviews of Interventions Version 6.2. Cochrane; 2021.

6. Cipriani A, Furukawa TA, Salanti G, Chaimani A, Atkinson LZ, Ogawa Y, et al. Comparative efficacy and acceptability of 21 antidepressant drugs for the acute treatment of adults with major depressive disorder: a systematic review and network meta-analysis. Focus 2018; 16: 420–9.

7. Grant MJ, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Infor Libr J 2009; 26: 91–108.

8. Levac D, Colquhoun H, O'Brien KK. Scoping studies: advancing the methodology. Implement Sci 2010; 5: 1–9.