Table 1. Definitions of the five Essential Areas of peer
review.
The team described best practice in each Essential Area in detail, using
the case studies and findings from the literature review, and developed
a checklist of questions to help journal teams. The work was shared as a
preprint in April 2018 (Allen et al ., 2018) and published after
peer review and revision (Allen et al., 2019).
At the same time a smaller group of colleagues created an online tool
for journals to use, the ‘Better Peer Review Self-Assessment, Version
1.0’, developed from the checklist presented in Appendix 1 of the
preprint. The Self-Assessment was built on Microsoft
FormsTM and Microsoft FlowTM, with a
dashboard built using Microsoft ExcelTM that partially
automates the creation of further feedback in the form of a Better Peer
Review Self-Assessment Quartile, Badge, and Data Visualization. Sixteen
colleagues attended three workshops in September 2018 in Wiley offices
in the USA (Hoboken) and UK (Chichester and Oxford). They completed the
Self-Assessment and shared their feedback, which led to improvements to
the Self-Assessment. This was then made available to Wiley colleagues as
Version 2.0 and is now accessible to everyone, including individuals
outside Wiley (
https://wiley.com/go/betterpeerreview).
The Better Peer Review Self-Assessment comprises three steps. First, in
the ‘Think and Reflect’ step, journals answer 48 questions focused on
the five Essential Areas. Journals decide how to approach the
Self-Assessment: individual team members can look at the questions prior
to undertaking the Self-Assessment as a group or, if preferred, an
individual journal team member such as a Managing Editor can first
complete the Self-Assessment, then discuss the results with the rest of
the team, revise practices accordingly, and then repeat. Journals must
answer the question and also briefly explain the rationale for their
answer with a free text summary. Next, in the ‘Immediate Feedback’ step,
the journal receives instant on-screen feedback as well as an immediate
record by email of their answers. Finally, in the ‘Summary Feedback’
step, journals receive another follow-up email with detailed information
on their Quartile compared with other journals that have completed the
Self-Assessment, Badge, Data Visualization, and some hints and tips for
the journals to use, if they wish, to improve their processes. The
‘Badge’ is a radar plot illustrating performance (Figure 1). The Data
Visualization breakdown is a histogram comparing a journal’s scores with
the mean scores reported by all other journals (Figure 2). The Better
Peer Review Self-Assessment therefore enables journal teams to identify
their strengths and weaknesses; to find out how their practices across
the Essential Areas compare with those of their peers; and to receive
guidance about how they might improve their processes.