Recently, I published a systematic review in Earth-Science Reviews along with my colleague Joris de Vente, in which we studied the impact of climate change on soil erosion, with an additional focus on conservation measures and land use change. This is the first systematic review that I have published, but it already gave me some interesting insights. In this blog post I want to share those insights, which hopefully can help you in your next systematic review.
First, I will give a short overview of our results. We collected more than 30 variables from 224 papers. This is actually where a systematic review differs from a normal literature review, you collect data from a large number of papers and use those data to draw your conclusions. The variables we collected can be subdivided over bibliographic variables, study area characteristics and, most importantly, methodological variables. We included a statistical analysis of the methodological variables and asked through an online questionnaire the opinion of other researchers on the robustness of these methodologies. From the results of the questionnaire we obtained robustness scores, which we translated to weights per study and determined weighted statistics. We showed that soil erosion is projected to increase under climate change, with an increasing trend towards the end of the century. Besides, we showed how land use change has positive and negative impacts and that conservation measures often mitigate the impacts.
Of course, there was a process that led to this systematic review, which started already in 2018. We initiated this research line with two studies focusing on methodologies used in climate change impact assessments, first focusing on bias-correction methods and climate model ensemble size and later on soil erosion model conceptualization. In both cases we had some thoughts about the most common methods used in this kind of assessments, but wanted to have a better foundation, which we obtained from the literature. So we started collecting data from SCOPUS, initiating the database that ultimately led to the systematic review. So, it is useful to start small, with some specific research questions, and work your way towards a systematic review in a few years. I think this worked very well for our case, because we could focus on these specific methodologies, meanwhile gaining a lot of knowledge on these methods, which helped us to better frame the systematic review later on.
I mentioned that we used an online questionnaire to ask fellow scientists their opinion about the methodologies used in soil erosion impact assessments. The information we obtained from the questionnaire we used to quantify the uncertainty of the methods and, ultimately, the uncertainty of the studies included in the systematic review. From the database we obtained the most common authors that published on the subject, with the idea that those authors should have some advanced knowledge on the subject. We selected 65 authors, from which 19 filled the questionnaire. It was a very useful and interesting experience to ask these authors for their opinion. So, the database can give you some useful information, which can help you to go deeper into the subject.
Systematic reviews often generate a lot of scientific impact (i.e. citations), but the knowledge you obtain can also be very valuable for other purposes. So, when I started collecting data in 2018, I very soon had the feeling that I was collecting some useful data. Around the same time, I found out about the peer-review process of an IPCC report (Climate Change and Land). I subscribed to join the review process, first of all, because I was curious about the review process, but also about the content. It turned out that there was one section about the impact of climate change on soil erosion. I reviewed mostly that specific section, making use of the data I obtained from the database. It turned out that the lead authors were thus pleased about my review that they asked me to become a contributing author, which was a very interesting experience, but more importantly, it allowed me to share my knowledge to a very wide audience. In a more recent IPCC report (Climate Change 2022: Impacts, Adaptation and Vulnerability), I also participated in the review process. This time, most of my suggestions I gave on a similar section on the impact of climate change on soil erosion, were included in the final version of the report. I also frequently use the database when reviewing a paper, for instance to give suggestions for studies in similar climate zones or in the same country, if such information is lacking from the paper. With this I want to show that your systematic review can be very useful for policy reports and in peer-reviews, and can have an impact beyond the scientific community.
This blog post is largely based on a presentation I gave during the workshop “How to write a review paper in Hydrology” (EGU 2022), which was organized by the Young Hydrological Society. Please watch my presentation below:
In this post I wanted to share my experience in systematic review writing. For me the most important things to consider are that: (1) You should start small, with some specific research questions, which might one day evolve into a systematic review. (2) The bibliographic information, such as authors and journals, can be used in many useful ways, such as in an online questionnaire. And (3) that you should not wait for the scientific impact (i.e. citations), but your results can be very useful for policy reports or in peer-reviews. I hope this blog post gave you some valuable tips and will help you in your next systematic review!