Methodological characteristics of eligible studies are of obvious relevance for the systematic review. Concretely, if eligible studies are methodologically flawed, the conclusions drawn from a systematic review might be compromised or even misleading. Thus, it is recommended to formally assess risk of bias of included studies. Bias refers to a systematic distortion of study results or conclusions, e.g., resulting in an under- or overestimation of an interventions effect size. For example, common sources of bias in both preclinical and clinical studies are absence of blinding or researchers or randomization. Studies which omit such rigorous research practices are thus at high risk of bias might be excluded from the analysis to probe the robustness of the conclusion(s) (see step 9).
A plethora of different risk of bias assessment tools has been developed for both preclinical and clinical applications. The following two tools are dedicated risk of bias assessment tools for animal systematic reviews:
- The SYRCLE risk of bias tool which is based on the Cochrane risk of bias tool.
- The consensus statement good laboratory practice in the modelling of stroke. Please note that we have developed a tool which enables partial automation of risk of bias assessment from animal research. Get in touch if you are interested.
Pitfalls
A common pitfall during this step is the selection of an inappropriate risk of bias assessment tool. Care should be taken to choose the proper tool. If in doubt, a systematic review methodologist should be consulted.
References and further reading
Egger, M., Higgins, J. P. & Smith, G. D. Systematic reviews in health research: Meta-analysis in context. (John Wiley & Sons, 2022).
Boutron, I. et al. Considering bias and conflicts of interest among the included studies. Cochrane handbook for systematic reviews of interventions, 177-204 (2019).
Sena, E. S., Currie, G. L., McCann, S. K., Macleod, M. R. & Howells, D. W. Systematic reviews and meta-analysis of preclinical studies: why perform them and how to appraise them critically. Journal of Cerebral Blood Flow & Metabolism 34, 737-742 (2014).
Hooijmans, C. R. et al. SYRCLE’s risk of bias tool for animal studies. BMC medical research methodology 14, 43 (2014). https://doi.org:10.1186/1471-2288-14-43