Before you start, ask yourself the following two questions:
Does my research question have contemporary relevance, e.g., to inform clinical decision making or animal experiment design?
Have other authors addressed or are planning to address my question using systematic review? Make a check on Pubmed and on the prospective registry for systematic review protocols PROSPERO. You can also check on the Open Science Framework.
A systematic review requires rigorous methodology. You can conduct your own systematic review by following these steps:
Having a specific research question is among the most critical steps during the systematic review process. Consider the following examples:
What is the effect of drugs on animal models of neurological disorders? What is the effect of drugs on animal models of acute stroke? What is the effect of fluoxetine on animal models of acute stroke? What is the effect of fluoxetine on functional outcomes in rodent models of acute stroke?
Ideally, a systematic review team comprises expertise from different fields. At a bare minimum, a team should include two researchers to enable independent abstract and full text screening as well as data extraction and risk of bias assessment to be in accordance with formal guidelines (see steps 5, 6, and 7). However, under ideal circumstances, the team should involve additional expertise:
Librarians/information specialists: they can help you to refine your search strategy and usually know a great deal about the plethora of biomedical databases which should be searched for the systematic review.
A priori writing and registration of a systematic review protocol is a key step for conducting a rigorous systematic review. This is important to mitigate a variety of biases during the review process. A pre-defined protocol also helps to structure the review process. Such a systematic review protocol pre-specifies the employed review methodology. Concretely, the protocol should predefine the following parameters:
Research question: What is the rationale and PICO of your study?
A good literature search for a systematic review should be both comprehensive and reproducible. With this, the focus of the search should be on sensitivity rather than on specificity, i.e., to not potentially miss relevant studies and condoning to also identify irrelevant studies (which will be sorted for eligibility during step 5). The search strategy should be meticulously documented; this is not only important for other groups to potentially reproduce the search but it is also key for potential later update searches.
The reference library identified by the comprehensive literature search will still comprise non-relevant references. Thus, these references must be screened for eligibility based on titles and abstracts. At least two reviewers ought to sort these references independently from each other (i.e., blinded to the decision of the other reviewer). Discrepancies between the two reviewers can either be resolved by discussion or by a third reviewer. Eligible references will be retrieved for full text analysis and potentially data extraction.
The full text of potentially eligible references will be scrutinized (as identified in step 5), again by at least two independent reviewers. This is particularly important for records in which the title and/or abstract provided ambivalent information on eligibility. EndNote provides a full text fetch function which is able to automatically find most of the full texts available through institutional licenses. Remaining full texts can be searched manually. A tool which renders manual full text search particularly quick is Google Scholar with its sensitive search engine, e.
For references which remain eligible at full text stage, data of interest will be extracted (or, as suggested in step 6, it is advisable in most cases to combine full text screening with data extraction). The data to extract should have been predefined in the protocol (see step 3). Of note, additional data may be extracted. Like abstract and full text sorting, data should be extracted by two independent reviewers. Covidence offers the option for custom-designing extraction sheets, based on a subscription-fee.
Methodological characteristics of eligible studies are of obvious relevance for the systematic review. Concretely, if eligible studies are methodologically flawed, the conclusions drawn from a systematic review might be compromised or even misleading. Thus, it is recommended to formally assess risk of bias of included studies. Bias refers to a systematic distortion of study results or conclusions, e.g., resulting in an under- or overestimation of an interventions effect size. For example, common sources of bias in both preclinical and clinical studies are absence of blinding or researchers or randomization.
After compiling the necessary data, the goal of this step is to answer the initially posed research question as defined in the protocol. For this, all review authors should come together, ideally with knowledge about the extracted data and the overall risk of bias of eligible studies. Several consensus rounds among study authors might be required to draw a conclusion.
If defined in the protocol, now is also the time to conduct a sensitivity analysis to determine the robustness of the conclusion(s).
It is critical to make your systematic review publicly available, independent from what the outcome was (and even if no clear conclusions could be made). The reason for this necessity is twofold: first, to mitigate publication bias and second, to inform colleagues about your study to avoid unnecessary duplication of work. Be aware that the status of your protocol should be updated upon publication of your manuscript.
Be cautious when interpreting your results and make sure to acknowledge sources of bias; consider heterogeneity, generalisability, and relevance.
