By: Daniele Wikoff, PhD, Toxicological Sciences Associate Editor, and Gary W. Miller, Toxicological Sciences Editor-in-Chief
We are all opinionated. We have our favorite musicians, artists, and foods. We can debate politics or fashion for hours on end. As scientists, we have our opinions about reagents, animal models, and methodologies. Diversity of opinions is one of the aspects of human nature which makes life interesting. However, when we must make decisions about the potential adverse effects of a chemical, our opinions are less valued. Indeed, our passion in our own research, our training, and our employers or sources of funding can introduce bias into our thinking. It is critical that we acknowledge such biases exist and that we approach science in ways which counter these inherent biases. This is why Toxicological Sciences has launched a new category of articles: Systematic Reviews.
Systematic reviews are used in many fields as a way of evaluating research across a particular topic in a systematic, transparent, and rigorous manner (Stephens et al. 2016). The classical narrative review leaves the assessment of the literature to the author and typically allows the authors' opinions and insights to drive the direction of the manuscript (Johnson and Miller, 2014). While the traditional narrative review serves an important role, as experts generally have important insights and we can learn from their opinions, many decisions in toxicology rely on a comprehensive and even-keeled analysis of the published literature. This is what systematic reviews are designed to do.
Components of a systematic review include problem formulation and protocol development, identification and selection of studies, critical appraisal of individual studies, integration and synthesis of the body of evidence, and reporting (Wikoff and Miller 2018). A systematic review is built around the objective, which takes the form of a “PECO” statement or question, typically constructed via an iterative practice by a multidisciplinary team. The PECO represents the population (P), exposure (E), comparator (C), and outcomes (O) of interest. Problem formulation aims to characterize the PECO and context around it, the protocol provides methods to evaluate the PECO, and the remaining elements involve the actual assessment and reporting of the PECO.
Thus, similar to how one may conduct an experiment, a systematic review defines the approach and parameters in advance, is carried out as prescribed, and reports well-documented findings (including any deviations, etc., along with the findings). Systematic reviews require a certain level of rigor of the underlying process and principles in the conduct of the review. Though systematic reviews do not eliminate the role of expert judgment, they can remove—or make more transparent—underlying biases. That is to say, they are not opinion pieces.
One of the great features of systematic reviews is the admission that scientists are human—that we have biases and opinions. Good for conversation, not so good for toxicology. The ethos of systematic reviews is one that we should embrace for all of our research. The challenges being addressed by the National Institutes of Health regarding rigor and reproducibility and previously addressed by Toxicological Sciences (Miller 2014) take a similar approach to systematic reviews. The goal is to produce work that is transparent, can be reproduced, and is accurate. This involves detailed reporting of the methods, providing access to the raw data, and explaining how conclusions were reached. Isn’t that what we are supposed to be doing as scientists in all of our work? Although systematic reviews were born out of a need of regulatory bodies, the spirit is one which should be welcomed by bench scientists: Show your data, explain your approach, and design studies which prevent the introduction of bias.
Trainees are encouraged to learn about systematic reviews. Even if one does not plan on pursuing this rather onerous approach to literature review, the structure is one which should be followed when reviewing the literature whether it be for a dissertation or manuscript. What is one’s expectation for good science? How does one define those parameters? We tend to be influenced by the names of the journals or authors when we should really be looking at the quality of the science. Systematic reviews attempt to define the standards of quality when evaluating research and, over time, will increase the quality of the conduct and reporting of toxicological studies. A difficult, but worthwhile task.
References
Johnson, J. A., and G. W. Miller. 2014. “Contemporary Reviews in Toxicology.” Toxicological Sciences 142: 4–5.
Miller, G. W. 2014. “Improving Reproducibility in Toxicology.” Toxicological Sciences 139: 1–3.
Stephens, M. L., K. Betts, N. B. Beck, V. Cogliano, K. Dickersin, S. Fitzpatrick, J. Freeman, et al. 2016. “The Emergence of Systematic Review in Toxicology.” Toxicological Sciences 152: 10–16.
Wikoff, D. S., and G. W. Miller. 2018. “Systematic Reviews in Toxicology.” Toxicological Sciences 162: 335–337.