ECR Reviewers Platform

Welcome

0.1 About this Project

The reproducibility crisis affects all of published research, in and outside of Switzerland. Early career researchers (ECRs) have been among the strongest advocates for transparency, reproducibility, and good scientific practices. However, their voices are still underrepresented in many parts of the academic field. At the same time, there is a growing demand for ECRs to peer review scientific papers. As the number of submissions of manuscripts to journals grows disproportionately faster than the number of new researchers, journal editors increasingly reach out to junior reviewers. However, peer review is still learned “on the job” which makes it especially challenging to ECRs. Our project aims, in a first place, at empowering junior reviewers and equipping ECRs with the necessary skills to conduct methodologically rigorous peer review. In the long term, we want to build a platform with and for ECRs to increase their representation in scientific expert boards and research evaluation panels in Switzerland.

0.2 Aim of this Guide

This guide contains practical advice on what to consider when being asked to peer review for a scientific journal; from deciding on whether or not to accepting a peer review invitation to what to consider after submitting the report. The guide will be used in a workshop and a platform to empower non-statistician ECRs when conducting peer review of quantitative and empirical research. Key points of good peer review are summarized with reference to open (educational) resources, publications, blog posts, checklists and tools for further reference. To help ECRs critically evaluate the validity of the presented statistics, we put special emphasis on the review of the statistical design and methods. Ensuring that only methodologically sound research is published is essential. However, there are many cases where statistical errors are overlooked. Possible reasons include a lack of statistical knowledge, the lack of clarity on what peer review should cover, or common misconceptions on statistical concepts. Our guide is meant to support non-statistician ECRs in promoting good research practices via peer review.

With this guide we specifically, but not exclusively, target non-statistician ECRs in empirical and quantitative research fields, such as psychology, economy, biology, or political science. The targeted ECRs are situated in a career stage between early doctorate and early post-doctorate, with only limited experience in peer review.

0.3 Structure of this Guide

The guide starts with some general information about the peer review and publication process, and the role of peer reviewers. Specific information is provided in “Info Boxes”, summarizing and defining relevant concepts, and “Tool Boxes”, aiming at giving practical advice and providing helpful tools to the reader. Both types of boxes include additional information, which can be skipped when going through the document. After providing guidance on how to answer to an invitation to peer review, we will support the reader through the different stages of peer review, including specific elements to look out for while reviewing each sections of the manuscript. The guide ends with instructions on how to write the report and further information on what may happen next.

Info Box

This is what an Info Box looks like - Info Boxes summarize and define relevant concepts.

Tool Box

This is what a Tool Box looks like - Tool Boxes provide the reader with practical advice and helpful tools.

Further Information

Within the guide, you will also find this sort of window with further information. Just like the Info and Tool Boxes, you can open and collapse these windows. That way, you can read them or ignore them according to your interests.

1 Acknowledgements

We want to thank the experts who were part of our online workshop on June 26, 2024 and gave valuable feedback on our guide: Gorka Fraga González, Gabriel Okasa, Karin Bütler, Lareunt Donzé, Alodie Rey-Mermet, Nicolas Rothen, Darías Holgado, Romain-Daniel Gosselin, Ana Marusic.

We received funding from the Swiss Academy of Sciences for this project. The present guide is the first output of this project.