Peer review

Providing guidance on the Cochrane Library editorial policy on peer review.

Basis of peer review

Peer review describes the objective evaluation of clinical and scientific research, usually (but not restricted to) before publication. 

Protocols for Cochrane reviews are peer reviewed to ensure that the research question is valid, the methods suggested are appropriate, and to avoid duplication of effort. Editors may use peer review reports to ensure that resources are allocated appropriately; for example, to prioritise protocols that answer the most relevant questions. 

Cochrane reviews are peer reviewed to ensure that they follow the published Cochrane protocol or that any deviation from the published protocol is sufficiently explained; to ensure the research question is still valid; to identify whether any relevant and important studies have been excluded, the clinical context is correct and up-to-date, the methodology is appropriate and the conclusions are based only upon the data available. 

Editorial roles during peer review: a general guide

Managing Editor

Managing Editors are responsible for managing the peer review process, including:

• Selecting and inviting suitable peer reviewers, following the requirements of Cochrane’s editorial policy on peer review ;
• For Diagnostic Test Accuracy (DTA) protocols and reviews, liaising with the DTA Editorial Team, to ensure that the DTA editorial process is followed;
• Keeping accurate records of the peer review process, including who has been invited and when, and recording all permissions and conflict of interest statements, compliant with the data protection policy;
• Providing deadlines for receipt of comments;
• Providing suitable materials to peer reviewers (e.g. guidance on peer review, peer review checklist, etc.);
• Ensuring that all peer reviewers have provided a 'Potential conflict of interest' statement;
• Sending reminders for late reports;
• Collating comments from peer reviewers;
• Handling all communication relating to the peer review process;
• Maintaining a list of the editorial group’s peer reviewers to be published on the group’s website on an annual basis;
• Collating and sending feedback to peer reviewers; and
• Raising any concerns with the editorial group’s Co-ordinating Editor or Executive Editor.

Executive Editor

The Executive Editor is responsible for oversight of the peer review process, including:

• Ensuring that peer review is conducted according to current Cochrane editorial policy;
• Ensuring that peer review procedures are up to date and published on the group’s website;
• Evaluating the final draft of the Cochrane review and ensuring that it is ready for external peer review;
• Suggesting appropriate peer reviewers;
• Reviewing the collated peer review reports and, based upon these reports, advising the Cochrane Review authors how to revise the manuscript;
• Ensuring that all relevant peer review comments have been addressed by the authors; and
• Raising any concerns with the Editor in Chief. 

Network Senior Editor [role duties need reassignment]

• Supporting CRG Co-ordinating Editors in making decisions relating to peer review.
• Adjudicating on matters relating to peer review.
• Raising any concerns with the appropriate external body; for example, institutions and funders.

Editor in Chief

• Ensuring that the Cochrane editorial policy on peer review is up to date and in line with international conventions;
• Adjudicating on matters relating to peer review.

Number and expertise of peer reviewers

Cochrane's editorial policy on peer review requires that every Cochrane review be peer reviewed by

• at least one clinical/topic specialist (with a minimum of one external to the editorial team), and
• one statistician/methodologist (who may, in some circumstances, be part of the editorial team).

It is also expected that at least one consumer peer reviewer be involved per Cochrane review.

The number of peer reviewers and the mix of expertise should be appropriate for the topic and complexity of the Cochrane review. For example, Cochrane reviews covering a multidisciplinary topic might need to seek several clinical/topic specialists. Peer reviewers with different skills should be guided to look at specific aspects of a review; for example, it might be appropriate to guide methodologists to comment only upon the methods section of a review).

Additional types of peer reviewers recruited to comment upon Cochrane reviews may include content specialists, systematic review specialists, or information specialists.

Required types of peer reviewers

• Clinical/topic specialist

This term is intended to describe anyone who has in-depth knowledge in the topic area covered by the review. In most cases, but not always, this will be a clinician. At least one subject-specialist must be external to the editorial team, i.e. they must not hold an editorial role with the Cochrane group handling the review. Fields are a potential source of subject peer review expertise external to the editorial team.

At protocol stage, the clinical/topic specialist should be able to assess whether the research question is valid.

At review stage, the clinical/topic specialist should be able to

• • • assess whether the research question is still valid (if it isn’t, what’s changed?);
    • identify whether any relevant or important studies have been excluded; and
    • determine whether the clinical context is correct and up-to-date.

• Statistician/methodologist

If the Cochrane review deviates from standard methods or uses complex methods, it should be peer reviewed by one statistician/methodologist. This person may have a role in the editorial team. If the Cochrane review uses standard methods, these can be checked by an appropriate member of the editorial team.

• Consumers

It is expected that editors seek involvement from consumers or other potential users of the Cochrane review. Consumer peer review ensures that Cochrane review questions are relevant to people requiring and accessing health care, and that meaningful outcomes and potential harms are considered.

Of particular importance is the role of consumers in the peer review of protocols for Cochrane reviews is to highlight or identify additional outcomes of importance. Consumer peer reviewers also check the language used in a Cochrane review, ensuring that the review is sensitive to consumers, and that medical terminology is used sparingly and jargon is explained wherever possible, so that Cochrane reviews can be read easily by a wide audience. Note that, in some cases, consumers are also topic specialists.

Cochrane’s Consumer Network recommends that editors use TaskExchange to recruit for suitable consumer peer reviewers. Also available are the ACTIVE project resources from the Cochrane Training website.

Additional sources of peer review expertise

• Fields

Cochrane Fields have access to methodologists, clinicians, policy-makers and consumers with expertise and experience relevant to their topic area. If a Cochrane review covers a topic relevant to the scope of a Cochrane Field, editors are advised to contact the Field to obtain a subject-specialist peer reviewer.

Obtaining peer review input from a Field can ensure that the review has addressed all methodological and clinical issues appropriately and has maximum relevance to, and impact upon, the external stakeholders in this area. 

• Study authors

When a topic field is small and it is difficult to find peer reviewers with expertise, it may be necessary for authors of trials or studies included in a Cochrane review (‘study authors’) to be invited to peer review. This is acceptable as long as at least one other peer reviewer is external to the editorial team and independent of any peer reviewer studies included in the review.

If a peer reviewer is an author of an included trial or study, this must be declared in the peer reviewer’s declaration of interest form and in the manuscript’s conflict of interest statement. Authors of the Cochrane review should also be made aware that a peer reviewer was a study author when receiving peer review comments.

Considerations for publication types

• Continuity from protocol to review

It is highly desirable to use the same peer reviewers for the protocol, the full Cochrane review, and any updates of the review. If this is not possible, it may be helpful to send key points from previous peer review reports to the current peer reviewers to ensure that time is not wasted revisiting previous decisions (for example, on the scope of the review).

• Peer review of Cochrane reviews of diagnostic test accuracy

The peer review process for Cochrane reviews of diagnostic test accuracy (DTA Reviews) is managed centrally by the Central Editorial Service to assure scientific quality and manage the limited peer review expertise for these reviews. See Internal and external review for diagnostic test accuracy reviews in the Editorial Manager Knowledge Base.

Exceptions to named peer review

Consumers

As set out in the editorial policy, consumer peer reviewers are exempt from the named peer review process and may choose to remain anonymous. Consumers must still follow the policy's requirement that peer reviewers declare any potential conflicts of interest every time they undertake peer review of a Cochrane review.

Approved by the Editor in Chief

If a Cochrane Review Group believes that the peer review process for a particular Cochrane review will be significantly negatively impacted by using named peer review, the Managing Editor or Co-ordinating Editor may apply to the Editor in Chief of the Cochrane Library for a policy exception on a review-by-review basis. In each case the Cochrane Review Group must provide details, including the reason for opting to remain anonymous. Any exemptions must be forwarded to the Cochrane Editorial & Methods Department (emd@cochrane.org).

Actions for editors

• Informing authors

Editors must inform the author team when an exemption is in place and that it has been approved.

• Informing the Editorial & Methods Department

Editors must forward any approved exceptions to the Editorial & Methods Department (emd@cochrane.org) with the following information:

• • • Review title
    • Date
    • Number of peer reviewers to remain anonymous
    • Reason(s) for opting to remain anonymous

• RevMan/Archie guidance: when peer reviewer remains anonymous to the authors

When peer reviewers return comments, editors should check that the peer reviewer has agreed to take part in named peer review. If the peer reviewer is remaining anonymous, the editors should update the peer reviewer’s Properties sheet in Archie as follows:

• • • Ensure the peer reviewer has been assigned a Group role of 'Peer reviewer'.
    • Create a new Administrative note with the Title e.g. ‘2018 closed peer review submitted’.
    • Add the name of the protocol/review to the Note text.

Editors can then search for all peer reviewers that have submitted closed peer review by using the Advanced Search in Archie as follows:.

• • • The People option should be selected at the top with the Match all rows (AND) option.
    • Role in Entity | Peer reviewer | [Insert Cochrane Review Group] | Active
    • Note Title | Contains | YEAR closed peer review submitted

Inviting peer reviewers

See Invite comments from editors or peer reviewers in Editorial Manager Knowledge Base for implementation information.

The invitation email should contain the following:

• The title and abstract of the Cochrane Review (or title only if a protocol);
• Deadline for return of comments;
• A statement that all of the information shared is confidential; and
• A link to the Cochrane conflict of interest policy.

The follow-up email (after acceptance of the invitation) could include the following:

• Published protocol (for reviews)
• Title proposal/registration form (for protocols)
• Full manuscript (note that this should only be shared with peer reviewers after the invitation to review has been accepted)
• Screening documents, if available
• Review-specific information, such as completed data extraction forms and copies of the included/excluded studies (for reviews).
• Peer review checklist
• Details of whom to contact with questions or concerns
• Specific peer reviewer guidance (only necessary if you are inviting the for the first time)
• A link to the Cochrane Library editorial policy on peer review, highlighting the section on 'Peer reviewer conduct'

Prospective peer reviewers should be informed that acceptance of the invitation to peer review for Cochrane is also taken as consent for peer reviewer details to be stored within the Cochrane peer reviewer database.

Using Cochrane Engage to find new peer reviewers

Cochrane Engage can be used to find new peer reviewers including consumer peer reviewers, clinical specialists, and methodologists, or to advertise new peer review tasks. For additional approaches for finding consumer peer reviewers, see Section 2.4 of the 'Standard Operating Procedure – Central Editorial Service Editorial Process' document.

Editors can point individuals interested in peer review opportunities towards Cochrane Engage to complete a profile. This is the route for potential peer reviewers being sign-posted on cochrane.org, Become a Cochrane peer reviewer.

Peer reviewers delegating their responsibilities

Cochrane recognizes the need for training and mentoring for early career researchers, and the delegation of peer review responsibilities to junior staff members is therefore permitted under certain circumstances, for example, when the junior staff member is a subject expert and the experience is used as a training or mentoring exercise.

The invited peer reviewer must request permission from the Central Editorial Service to delegate their responsibilities, and must provide the full name and contact details (including position and institutional email address) for the junior staff member so that they can be acknowledged appropriately. The invited peer reviewer must sign-off on, and take responsibility for, the final peer review report. Both the original peer reviewer and the delegated peer reviewer must return the “Potential conflicts of interest” and the “Permissions” statements from the peer review checklist; they may both be acknowledged in the permitted ways for their contributions to the peer review report. 

Peer reviewer declarations of interest

See Cochrane's Conflict of Interest Policy (2020), Section 6.3 Rules for peer reviewers, for rules relating to declared interests and for editors' responsibilities in assessing declared interests.

Following peer-review

Collating peer review comments

Peer review comments should be collated before sending to the authors. Editors should ensure that names and affiliations are included for those participating in named peer review, and removed for those not participating in named peer review (see above, Exceptions to named peer review).

Ensuring that authors address peer review comments

The editor with sign-off responsibility for the Cochrane Library article is responsible for ensuring that authors have considered and incorporated any relevant and reasonable peer reviewers' comments that they receive.

Editors are under no obligation to undertake any further editorial steps until they are able to check the revised version of the article meets the requirements of the peer reviewers.

Feedback to peer reviewers

Providing comments and feedback to peer reviewers should improve the quality of peer review reports over time. Feedback is particularly valuable to new peer reviewers and may be an incentive to continue the relationship with Cochrane. For consumer peer reviewers, editors might consider implementing a mentoring system, whereby experienced consumer peer reviewers give support to new consumer peer reviewers.

When requested, editors should provide peer reviewers with a copy of the authors’ responses to their peer-review comments.

Acknowledging peer reviewers

Acknowledgement in the published article

Peer reviewers should be offered the option of acknowledgement in the article to which they contributed. Include text in the Acknowledgements section of the Cochrane article; for example:

The authors [or the editors] are grateful to the following peer reviewers for their time and comments: [Insert peer reviewer names and affiliations].

For reviewers who choose not to be publicly acknowledged, make clear how many peer reviewers were involved in the process, for example:

The authors [or the editors] are grateful to the following peer reviewers for their time and comments: [insert peer reviewer names and affiliations], and also to the [insert number] peer reviewers who chose not to be publicly acknowledged.

Other forms of acknowledgement

• Payment

Peer reviewers may, in some circumstances, receive payment from Cochrane Review Groups; for example, for undertaking a rapid review or when specialist/expert input is needed. This is at the discretion of the CRG, who will fund this payment.

• Certificates

Certificates are appreciated by some peer reviewers who are able to use the recognition of peer review to count towards their continuing professional development (CPD) or continuing professional education (CPE). Editors may wish to generate certificates for each completed peer review report; on an annual basis; or to highlight the contributions of particularly frequent peer reviewers (e.g. the top five peer reviewers in a calendar year).  

• Cochrane Membership

Peer reviewers can also earn Cochrane Membership through completing peer review tasks. 

• Publons

Publons is a service that provides credits for peer review in a format that can be used in CVs and funding applications. See the Publons website for more information. 

Cochrane peer reviewers can set up an account on Publons and then forward their peer review acknowledgement emails from Editorial Manager to Publons so that their peer review activity can be recorded to their profile.

Note that Publons has the facility to upload full peer review reports. As peer reviewers of Cochrane reviews are bound to confidentiality until the Cochrane review is published, peer review reports of Cochrane reviews that have yet to be published should not be posted to Publons. Any confidential comments to the editor should also not be posted on Publons.

• ORCID

ORCID provides a unique, persistent digital identifier to all registered researchers to support automated links between the researcher and their professional activities to ensure that work is recognised and attributed correctly. Peer review activity can be recorded in an ORCID account, either via a service such as Publons, or by being added manually.

Support resources

Committee on Publication Ethics: Peer review processes 

ICMJE recommendations: Responsibilities in the submission and peer review process, including responsibilities of peer reviewers 

Back to Pre-publication processes  •  EGR homepage