For references which remain eligible at full text stage, data of interest will be extracted (or, as suggested in step 6, it is advisable in most cases to combine full text screening with data extraction). The data to extract should have been predefined in the protocol (see step 3). Of note, additional data may be extracted. Like abstract and full text sorting, data should be extracted by two independent reviewers. Covidence offers the option for custom-designing extraction sheets, based on a subscription-fee. Otherwise, Google Sheets provides a free web-based alternative, with online spreadsheets for independent data extraction. In the spreadsheets, rows constitute individual references for data extraction whereas columns constitute individual parameters to extract.
Besides abstract screening, data extraction is among the most labor-intensive systematic review tasks. Artificial intelligence has also been leveraged to extract specific data—such as the study population, intervention, outcome measured, and risks of bias—from abstracts or full texts. Note that although these methods are not yet at a level appropriate for the evaluation of individual publications, they are considered suitable for deployment on larger reference libraries (>1000 records) in a research-improvement context.
Numerical data should in first priority be extracted from text or tables and in second priority from graphs, e.g., by using tools such as the Adobe Desktop Ruler or WebPlotDigitizer. You might also need to convert units if different measurement scales are used across different studies.
Pitfalls
There are two common pitfalls during this step:
- First, it is a common mistake to extract as much parameters as possible. It is advisable to focus on a smaller set of parameters to extract since time expenditure quickly escalates with an increasing amount of data to extract.
- Second, it is important to clearly characterize each parameter to extract using meta-data. Unequivocal definition of parameters can otherwise lead to confusion during later time points, e.g., when doing revisions on a paper. Also, similar to the abstract screening, we recommend a pilot extraction round to potentially refine the extraction template and identify potential uncertainties among reviewers.