What are the challenges of the academic peer review system?

The open research movement stands as a beacon for transparency and accessibility in the realm of academic inquiry and data dissemination. Its impact is profound, ushering in transformative practices such as open access, open data, open infrastructure, and open research methodologies that have permeated mainstream scientific discourse.

Peer review is a fundamental element of academic research and publishing, with a firm reliance on the global scholarly community to perform gatekeeping and filtering processes in pursuing high-quality and high-value scholarly publications: the “gold standard” in academic publishing [1].

In alignment with the overarching goal of transparency, there arises a critical need to spotlight open peer review—an increasingly influential process within scholarly circles. Recognizing the manifold benefits of peer review, numerous scientific publishers are embracing open peer review as a means to assess the integrity and excellence of submitted manuscripts.

This burgeoning trend underscores a collective commitment to fortifying the foundations of scholarly communication, ensuring that the rigorous evaluation of research findings is conducted in a transparent and accountable manner. As the open research movement continues to evolve, the integration of open peer review serves as a pivotal step towards fostering trust, collaboration, and innovation within the scientific community.

What is open peer review?

Peer review stands as a cornerstone of scientific research and communication, serving to authenticate and elevate published resources. A survey conducted by the Publishing Research Consortium revealed that approximately 82% of researchers acknowledge the indispensability of peer review in maintaining control over scientific discourse.

Through meticulous examination of various peer review methodologies, Tony Ross-Hellauer, head of the Open and Reproducible Research Group (ORRG) at Graz University of Technology, Austria, has formulated a comprehensive definition of open peer review. This concept, termed Open Peer Review (OPR), encompasses a diverse array of approaches aimed at fostering open science practices, as highlighted in Ross-Hellauer’s systematic analysis. Open peer review (OPR) is a cornerstone of the emergent Open Science agenda. Yet to date no large-scale survey of attitudes towards OPR amongst academic editors, authors, reviewers and publishers has been undertaken [2].

What are the challenges of an open peer review system?

The notion that transparency in author and reviewer identities leads to bias and retaliation is often debated. This concern is particularly prominent among junior scientists who are early in their careers. They may fear potential repercussions for critiquing well-established senior scientists upon whom they rely for career advancement opportunities.

This misconception has led to a perceived necessity for anonymity in the peer review process. However, it is important to recognize that blinded review does not guarantee protection against harassment. Research indicates that blinded reviews are ineffective in a significant proportion of cases, with failure rates ranging from 10 to 32% according to various studies.

The primary goal of Open Peer Review (OPR) is to instill integrity throughout the scholarly publication process. By embracing transparency, the open peer review system seeks to mitigate concerns about potential reprisals, thus fostering a more equitable environment. Any instances of misconduct are subject to public accountability and scrutiny within this framework. Apart from the challenges faced by OPR, OPR is seen as a sound reliable science for academe. OpenAIRE survey (2017) revealed that majority of the respondents favor OPR becoming mainstream scholarly practices of Open Science [3].

Ethical and policy guidelines for peer review play a critical role in mitigating instances of harassment and misconduct. Several reputable organizations and institutions, including COPE (Committee on Publication Ethics), the Council of Science Editors, the International Committee of Medical Journal Editors (ICMJE), and Wiley, have developed comprehensive guidelines specifically tailored to peer review practices. These guidelines serve as essential frameworks for maintaining integrity, professionalism, and fairness throughout the peer review process, ultimately contributing to the credibility and trustworthiness of scholarly publishing.

In Open Peer Review (OPR), the review report submitted by a junior researcher is made accessible to the public, allowing anyone to view and evaluate it. Consequently, when a reviewer provides insightful and constructive criticisms, it serves as a means to enhance their credibility within the research community. This transparency not only underscores the reviewer’s commitment to scholarly rigor but also showcases their capacity to contribute meaningfully to the advancement of knowledge.

Open Peer Review (OPR), while offering transparency and potential benefits, also faces several challenges.

Let’s explore these challenges:

1. Fear of Retaliation:
   • Transparency in author and reviewer identities can create bias and fear of reprisals.
   • Junior scientists, especially at the beginning of their careers, may hesitate to provide critical reviews due to concerns about negative consequences.
2. Unwillingness to Accept Signed Peer Review:
   • Historically, reviewers were often unwilling to review openly and have their reports published.
   • The shift toward signed peer review requires overcoming this reluctance.
3. Quality of Peer Review Reports:
   • Open peer review doesn’t guarantee better quality reviews.
   • Ensuring rigorous, constructive, and helpful feedback remains a challenge regardless of transparency.
4. Balancing Transparency and Anonymity:
   • While transparency is essential, maintaining some level of anonymity protects reviewers from potential backlash.
   • Striking the right balance is crucial to encourage honest and constructive reviews.
5. Variability in Implementation:
   • Different journals adopt varying forms of open peer review.
   • The impact of each attribute (e.g., publishing review content, sharing identities) differs, and finding the optimal approach is challenging.

In summary, while open peer review enhances transparency, addressing these challenges ensures its successful implementation and benefits for the scholarly community.

References

1. Gonzalez, P., Wilson, G. S., & Purvis, A. J. (2022). Peer review in academic publishing: Challenges in achieving the gold standard. Journal of University Teaching & Learning Practice, 19(5), 01.
2. Ross-Hellauer, T., Deppe, A., & Schmidt, B. (2017). Survey on open peer review: Attitudes and experience amongst editors, authors and reviewers. PloS one, 12(12), e0189311.
3. Wusu, O. (2018). Open science: a review on open peer review literature. Library Philosophy and Practice.