This document provides guidance on how to implement the Cochrane Policy for Managing Potentially Problematic Studies which details what Cochrane Review authors and Editors should do when serious concerns are raised about the trustworthiness of an included study or a study that may be eligible for inclusion in a Cochrane Review. This includes:

studies that have been formally retracted
studies that have a published Expression of Concern
studies with no formal post-publication amendment, but where Cochrane Review authors or Editors have concerns about the trustworthiness of the study data

Such issues in included studies may arise due to scientific misconduct, however untrustworthy study data does not necessarily occur as a result of misconduct and may be the result of poor research practices or honest error.

1 Searching for post-publication amendments

Cochrane Review Groups (CRGs) must ensure that the search strategy for a Cochrane Review captures any post-publication amendments published on included or eligible studies. Post-publication amendments include expressions of concern, errata, corrigenda and retractions.

Further guidance on how to include post-publication amendments in search strategies is available in:

2 Actions

Although it is not within Cochrane’s remit to conduct a formal investigation or arrive at a formal conclusion regarding the misconduct of an individual (or group of individuals), or about whether a published article should be retracted, Cochrane has a responsibility not to include studies on which there are serious concerns about the trustworthiness of the data. CRGs are expected to take steps to minimise the risk of including problematic studies in Cochrane Reviews, such as checking for post-publication amendments and acting on any concerns raised by review authors, peer reviewers, Editors, or readers about included studies.

This section details what should be done where there are concerns about the trustworthiness of the data in a study that is included, or is eligible for inclusion, in a Cochrane Review. These actions will differ depending on whether the potentially problematic study has a post publication amendment and whether the Cochrane Review is unpublished or published.

2.1 What to do when a study is retracted

Retracted studies should not be included in a Cochrane Review. See section 1 Searching for post-publication amendments for links to resources on how to avoid inclusion of retracted studies (see MECIR C48).

2.1.1 Resolution for an unpublished Cochrane Review (included study retracted)

The retracted study should be listed in the “excluded studies” section, with the reason noted, and including a citation to the published retraction notice. This reason must be noted using neutral language and should not accuse authors of scientific misconduct. If there is any doubt about the phrasing of such a note, the CRG must contact the named Associate Editor (who will refer on to the Research Integrity Team) before publication of the Review.

2.1.2 Resolution for a published Cochrane Review (included study retracted)

If an included study is retracted after the Cochrane Review has been published, the study data must be removed from all analyses and the extent of changes required in response to the retraction should be determined. While this takes place, an Editorial Note should be added to the Cochrane Review to clearly alert the reader that an included study has been retracted. The wording of the Editorial Note should be agreed with the EMD before publication and should cite the retraction notice. The CRG should contact the named Associate Editor (who will refer on to the Research Integrity Team as needed). See section 7 .4 Template text for Editorial Note on a Cochrane Review for template text.

Depending on the extent of changes required in response to the retraction(s), it might be appropriate to withdraw (retract) the Cochrane Review. The withdrawn notice should include a complete listing of the studies retracted, including citation to the retracted studies, and an indication of whether the intention is to update and republish the Cochrane Review or not. See Cochrane’s withdrawal policy for further information on what to include in withdrawn notices.

If the Cochrane Review is determined not to require withdrawal, a citation to the published retraction notice (of the included study) should be added to the text of the Cochrane Review as one of the references under the study ID, resulting in the publication of a new citation version of the review (using the “What’s new” events “Amended” AND EITHER “New citation: conclusions not changed” OR “New citation: conclusions changed”, as appropriate). See the What’s new publishing events policy for further information.

2.2 What to do when there is a published Expression of Concern

In cases where a journal is aware of concerns regarding a published article, an Expression of Concern may be published to alert readers to these concerns while an investigation takes place to determine whether the findings of the study can be relied upon or whether further action is needed, such as retraction or correction.

Where there is an Expression of Concern published on an included study, or a study that is eligible for inclusion in a Cochrane Review, the results of that study should be treated with a high degree of caution. The CRG Editor may wish to consider contacting the journal editor for additional information, however journals are often unable to give further information that is stated in an Expression of Concern while an investigation is ongoing. If further information is sought from a journal, all correspondence must be kept neutral and should follow the provided template provided in section 7.1.1 Asking a Journal Editor for further information about an Expression of Concern. All communication and information disclosed by journal editors must be treated confidentially by Cochrane authors and CRG Editors.

Depending on the reason for the published Expression of Concern, it may be appropriate to exclude the study or its data from the Cochrane Review (e.g. if the Expression of Concern is for serious concerns about the validity of the data). Note that Expressions of Concern may be published for reasons that do not affect the validity of the data (e.g. authorship disputes), in which case it may be considered appropriate to include the study in the Cochrane Review. If uncertain, contact the named Associate Editor (who will refer on to the Research Integrity Team as needed).

2.2.1 Resolution for an unpublished Cochrane Review (published Expression of Concern)

Based on the reason for the Expression of Concern, a decision may be made to include or exclude data from that study until such time that either the study or the Expression of Concern itself is retracted, or further evidence comes to light.

Depending on which decision is made, the following actions should be taken:

If the study is to be included in the Cochrane Review, the Review must include a citation to the published Expression of Concern as one of the references under the study ID, and a note must be added to the Cochrane Review to reference the Expression of Concern, together with the reasons for including the study (e.g. in the description of studies, results, discussion and abstract).
If the study is deemed to be eligible for inclusion in the Review but there are legitimate reasons to await the resolution of the outstanding Expression of Concern the study should be listed under studies “Awaiting Classification” with the citation to the published Expression of Concern.
If the study is to be excluded from the Cochrane Review, the reason for exclusion should be stated in the Excluded Studies table and the Expression of Concern noted.

Following investigation by the Journal or study author’s institution, the study may subsequently be retracted, or the Expression of Concern itself retracted. If this happens after the Cochrane Review has been published, the authors of the Review should follow the actions in section 2.2.3 What to do if the study or Expression of Concern is subsequently retracted.

2.2.2 Resolution for a published Cochrane Review (published Expression of Concern)

If the Expression of Concern relates to the validity of the data in the included study, an Editorial Note should be added to the Cochrane Review to acknowledge that concerns have been raised about one or more studies and/or study data included in the Cochrane Review. This should remain in place until either the Expression of Concern on the included study or the included study itself is retracted. The text of the Editorial Note on the review should be agreed with the named Associate Editor, who will refer to the Research Integrity Team, and should cite the Expression of Concern that is published on the included study. See section 7.4 Template text for Editorial Note on a Cochrane Review for template text.

A citation to the Expression of Concern published by the journal on the included study must be added to the text of the Cochrane Review as one of the references under the study ID, and a note must be added to the Cochrane Review to reference the Expression of Concern (e.g. in the description of studies, results, discussion and abstract). Note that this will require a “What’s new” event “Amended” but does not require a new citation version of the review to be published – see the What’s new publishing events policy for further information.

It may then be appropriate to analyse the effect of removal of the study/studies from the Cochrane Review. Depending on the outcome of this analysis, the following steps should be taken:

If the conclusions would not change, this information should be added to the Editorial Note on the Cochrane Review
If the conclusions would change, the Editorial Note on the Cochrane Review should be updated to include this information and to state that the Review will be updated.

2.2.3 What to do if the study or Expression of Concern is subsequently retracted

Following investigation by the Journal or study author’s institution, the included article with the Expression of Concern may be retracted, or the Expression of Concern itself may be retracted (indicating that the concerns about the study have been dismissed). In such a situation, the Cochrane Review authors should re-assess the potential inclusion of the study in the Cochrane Review and the following actions should be taken:

If the study is deemed eligible for inclusion in the Cochrane Review, the study should be included in the Review, and the Editorial Note on the Cochrane Review should be updated to state that the Expression of Concern on the included study has now been removed. The Editorial Note can be removed from the Cochrane Review at the next update.
If the study is retracted, the actions outlined in section 2.1 What to do when a study is retracted should be followed.

2.3 What to do if you have concerns about an included study

In some cases, Cochrane Review authors or CRG Editors may have serious concerns about an included study (see section 7.3 Examples of serious concerns about the trustworthiness of a study), or a study that is eligible for inclusion in a Cochrane Review, but there is no formal post-publication amendment published for that study.

At present there is no standard definition of, nor validated method to identify, a problematic study. As such evidence becomes available and consensus emerges in this area, this guidance will be updated. Until such a time, the following steps should be taken:

Check that there are no post publication amendments published about the study. Search for the study in PubMed, Medline (and other bibliographic databases, as appropriate), the Retraction Watch Database, and PubPeer to clarify if there are any Expressions of Concern, comments, or Letters to the Editor regarding the study, and to confirm that the study has not been retracted. See section 1 Searching for post-publication amendments and consult with an Information Specialist if further assistance is needed with how to do this.
If an Expression of Concern is identified, the actions outlined in section 2.2 What to do when there is a published Expression of Concern should be followed.
If the study has been retracted, the actions outlined in section 2.1 What to do when a study is retracted should be followed.
Check the version of the article on the publisher’s website, for any un-indexed notifications, e.g. ‘Editorial Note’ or links to post-publication amendments that do not (yet) appear on mirror sites/bibliographic databases.
If information is missing from the report of the included study and/or further information is required to determine whether you have concerns, contact the authors of the study to ask for clarification and/or unpublished information in line with MECIR standard C49. All correspondence must be kept neutral. Do not accuse authors of misconduct, or fabrication or falsification of data.
Describe your concerns in detail, including the exact method(s) used to determine that there may be a problem with the study, in an email to the Journal Editor using neutral language. You may find that the act of drafting this email crystalizes whether you have significant concern about the study if you are unsure. All language used must be kept neutral and follow the templates provided in section 7.1 Templates for corresponding with Journal Editors and authors. No accusations of scientific misconduct must be made. You should inform the named Associate Editor, who will refer to the Research Integrity Team, before contacting the journal editor. This allows the Research Integrity team to keep a track of such cases which will inform updates to this guidance. It is the responsibility of the Journal Editor to initiate further investigation (for example contacting the Author’s institution).
If you are unsure whether you have sufficient concerns about an included study to warrant contacting the Journal Editor, you may wish to consider following some of the steps described in section 7.2 Methods for determining whether you have concerns about a study. Please note however that these methods are all unvalidated, and do not necessarily indicate that a study is untrustworthy. They may, however, help to consolidate, strengthen, or alleviate your concerns about a particular study.

2.3.1 Resolution for an unpublished review (concerns about an included study)

If, after attempting to draft a letter to the Journal Editor as described in section 7.2 Methods for determining whether you have concerns about a study, you decide that you do not have sufficient concerns about the study to contact the Journal Editor, the study can be included in the Cochrane Review. No further action is needed.

If concerns remain and you contact the Journal Editor to request an investigation and/or clarification of whether the data can be included in a Cochrane Review (which can take a considerable time where a Journal and potentially also an institution is involved) the publication of a new or updated Cochrane Review should not usually be delayed. If you have sufficient concerns about an included study to warrant contacting the Journal Editor, the study of concern should be categorized as “Awaiting Classification”, with a note added to explain why. The language of the note must be kept neutral, for example “The data could not be verified”. If any other note of explanation is required, you must contact the named Associate Editor, who will refer to the Research Integrity Team.

There is no time limit on how long studies can remain in the “Awaiting Classification” category. It may be appropriate during protracted investigations to prompt the Journal Editor, as appropriate, for updates.

See section 2.3.3 What to do when a journal investigates for steps to take once an investigation is completed.

2.3.2 Resolution for a published review (concerns about an included study)

If after your own assessment (which may include following the steps described in section 7.2 Methods for determining whether you have concerns about a study), or attempting to draft a letter to the Journal Editor, you decide that you do not have sufficient concerns about the study to contact the Journal Editor, the study can remain included in the published Cochrane Review. No further action is needed. If the concern was raised by someone outside of the review author group or Editorial Team, they should be informed of this outcome.

If concerns remain and you contact the Journal Editor to request an investigation and/or clarification to determine whether the data can be included in a Cochrane Review (which can take a considerable time where a Journal and potentially also an institution is involved), an Editorial Note should be added to the Cochrane Review to state that the journal editor has been contacted with concerns about the included study. The statement should remain as neutral as possible and include links to any published articles that might provide additional information (e.g. Letter to the Editor, etc.) and propose a timeline for any update (if known). The text of any Editorial Note must be agreed with the Research Integrity Team before publication. You should contact the named Associate Editor who will refer to the Research Integrity Team.

After contacting the Journal Editor, the steps described in section 2.3.3 What to do when a journal investigates should be followed once the journal agrees to investigate the concern.

If the Journal Editor does not respond when concerns are raised, and no response is received on following up with the Journal Editor, consider contacting the Journal Publisher. If no response is received from the Journal Publisher, contact the named Associate Editor who will refer to the Research Integrity Editor for advice.

2.3.3 What to do when a journal investigates

Once you have contacted a Journal Editor, they should follow COPE guidelines to investigate the concerns raised to them. If concerns relate to trustworthiness of the data in a study, the Journal Editor may contact the author’s institution to ask them to conduct a formal investigation into the study data. This may take some time and the Journal may decide to publish an Expression of Concern to alert readers to the concerns while an investigation is underway. If this occurs, the Editorial Note on the Cochrane Review should be updated to cite the Expression of Concern on the article and the steps described in section 2.2.2 Resolution for a published Cochrane Review (published Expression of Concern) should be followed.

If the Journal Editor confirms that an investigation will take some time, it may be appropriate to analyse the effect of removal of the study/studies from the Cochrane Review. Depending on the outcome of this analysis, the following steps should be taken:

If the conclusions will not change, this information should be added to the Editorial Note on the Cochrane Review
If the conclusions will change, the Editorial Note on the Cochrane Review should be updated to include this information and to state that the Review will be updated. The Review should then be updated to include this information.

Following completion of a journal and/or institutional investigation, the concerns may be upheld or dismissed, and the journal may take one the following actions:

The Journal Editor may take no further action
The article may be retracted
A Correction article may be published as a post publication amendment to the journal article

If the Journal/Institutional Investigation deems that the study is trustworthy and decides to take no action, the study should be included in the Cochrane Review. In the case of a published Cochrane Review where the study was moved to “Awaiting Classification”, a ‘What’s New event’ must be added to indicate the change (i.e. the full inclusion of a study previously awaiting classification). See ‘Assigning What’s New events to Cochrane Reviews’ for details on actions that need to be taken when a study is added to a Cochrane Review. If an Editorial Note was added to the Cochrane Review, this should be revised to state that the Expression of Concern on the included study has now been removed. The Editorial Note can be removed from the Cochrane Review at the next update.

If the article is retracted, the steps detailed in Section 2.1 What to do when a study is retracted should be followed.

If a Correction article is published on the included study, the effect of this on the Cochrane Review should be determined and the Review updated if the correction affects the outcome of the review. If an Editorial Note was added to the Cochrane Review, this will need to be updated as appropriate. The named Associate Editor, who will refer on to the Research Integrity Team, should be contacted to agree the wording.

3 Recording information in CRS/CENTRAL

This section is relevant to Cochrane Information Specialists (CISs) and others who have a responsibility for searching bibliographic databases, importing records into the Cochrane Register of Studies (CRS), and adding eligible studies to the Cochrane Central Register of Controlled Trials (CENTRAL).

3.1 Importing records of retracted studies into the CRS

When a retraction notice relevant to a study that is already included in the CRS (or that will be imported into the CRS) is identified, it should be imported into the relevant CRS segment. Please refer to the HarmoniSR Guidance for formatting core (or mandatory) fields in reference and study records in the CRS.

3.2 Linking retraction notices in the CRS

CISs should link imported retraction notices to the CRS records for the retracted studies. For further guidance on how to do this, please refer to the Informational Specialists Portal on the Cochrane Community site and the Quick Reference Guide: How to deal with retractions in CRS.

3.3 Retraction notices in CENTRAL

Retracted studies that are eligible for CENTRAL (e.g. randomized and quasi-randomized controlled trials) should be added to CENTRAL via CRS. Retraction notices for CENTRAL study records should also be added to CENTRAL. Linking the retraction notice to the article being retracted using the CRS (see 3.2) will ensure that both the retraction notice and the article being retracted are visibly linked together in CENTRAL.

Sample CENTRAL records showing how retractions should appear in Cochrane Library:

Retraction notice in CENTRAL: https://www.cochranelibrary.com/central/doi/10.1002/central/CN-00145686/
Retracted CENTRAL record: https://www.cochranelibrary.com/central/doi/10.1002/central/CN-00158198/

4 Support for CRGs and Cochrane Review authors

Support and advice are available to CRGs from the EMD on managing cases where there are concerns about the trustworthiness of an included study, or a study that is eligible for inclusion in a Cochrane Review. You can access this support by contacting the named Associate Editor who will refer to the Research Integrity Team as needed. They will advise on the wording of communications to authors, journal editors, institutions and also text within a Cochrane Review (for example, explaining why data from a study were excluded from the analyses, or why a study is “awaiting classification”) to mitigate the risk of defamation/libel. If further legal advice or support is required, it can be arranged by the EMD.

5 Acknowledgments

The first version of this policy and guidance was drafted following discussions with members of the members of the Cochrane scientific misconduct policy advisory group (Andrew Moore, Ian Roberts, Emma Sydenham, Alison Avenell, Gerben ter Riet, Fergus MacBeth, Angela Webster, Brian Stafford, John Carlisle, Mbah Patrick Okwen, Anne-Marie Stephani, Joshua Cheyne, Gerry Stansby). The version has been revised following wider consultation with the Cochrane Community. The final policy statement and implementation guidance do not necessarily reflect the views of all participants of the advisory group.

6 References

Carlisle, J. B., Dexter, F., Pandit, J. J., Shafer, S. L. and Yentis, S. M: Calculating the probability of random sampling for continuous variables in submitted or published randomised controlled trials. Anaesthesia 2015 70(7): 848–858

International standards for clinical trial registries – 2nd edition. Geneva: World Health Organization; 2018. Licence: CC BY-NC-SA 3.0 IGO.

7 Appendix

The appendix contains the following sections:

7.1 Templates for corresponding with Journal Editors

All correspondence must be kept neutral and the following templates should be followed. If a template is deemed inappropriate for a particular situation or the CRG would like to deviate from the template, the named Associate Editor must be contacted, and will refer to the Research Integrity Team, before contacting the study author(s) or Journal Editor. Correspondence must never accuse individuals of misconduct or fraud.

7.1.1 Asking a Journal Editor for further information about an Expression of Concern

If an Expression of concern has been published on an included study, or study that is eligible for inclusion in a Cochrane Review and the CRG editor decides to ask the Journal Editor for further information, the following template should be followed. For further information, see section 2.2 What to do when there is a published Expression of Concern.

Dear <Journal Editor>,

My name is <name> and I am <role> at <CRG group name>.

I am writing to you about the following article published in the <journal>: <citation, including DOI> which has an Expression of Concern Published.

[For an unpublished Cochrane Review:]

[OR for a published Cochrane Review:]

<The article has been included in a published Cochrane Review <insert citation/link to published Cochrane Review>. In order to determine the effect this Expression of Concern has on the review, and whether we need to take any action, we would be grateful if you were able to provide any further information about the concerns relating to this study that led to the publication of the Expression of Concern.>

In addition, I would be grateful if you could answer the following questions: [insert any specific questions for the editor, for example:]

I thank you for your time and look forward to hearing back from you.

Best wishes,

7.1.2 Alerting a Journal Editor to concerns

If you have concerns about the trustworthiness of an included study, the following template should be used when contacting the Journal Editor for further information. For further information, see section 2.3 What to do if you have concerns about an included study.

Dear <Journal Editor>,

My name is <name> and I am <role> at <CRG group name>.

I am writing to you about the following article published in the <journal>: <citation, including DOI>.

[For an unpublished Cochrane Review:]

[OR for a published Cochrane Review:]

<The article has been included in a published Cochrane Review <insert citation/link to published Cochrane Review> and we have subsequently identified the following concerns with this article:>

[Insert factual, detailed description of your concerns regarding the article, including the exact method the author / review group used to decide there was a potential problem with the study. Example text to be expanded on with specific concerns/method of identification:]

As Editor of the Journal publishing this article, please could you investigate these concerns in line with guidelines from the Committee on Publication Ethics (COPE).

I thank you for your time and look forward to hearing back from you.

Best wishes,

7.2 Methods for determining whether you have concerns about a study

There is currently no standard definition of, nor a validated method to identify, a problematic study that has not otherwise been identified as such through a post-publication amendment. If you have serious concerns about an included study, you should ask the Journal Editor to investigate (see section 2.3 What to do if you have concerns about an included study). The following list provides options that others currently use when evaluating a study about which they have doubts about the trustworthiness. You may wish to consider following some or all of these steps before contacting the Journal Editor, either to help you define what your concerns are about the study or to determine whether you do have concerns about a study where you are unsure. None of the actions listed are validated ways for determine whether a study is trustworthy, and do not on their own indicate untrustworthiness of a study. You may wish to also look at section 7.3 Examples of serious concerns about the trustworthiness of a study.

Consider assessing the study against an (unvalidated) checklist such as the Trustworthiness Screening Tool (TST) developed by the Cochrane Pregnancy and Childbirth Group or the REAPPRAISED checklist for evaluation of publication integrity.
If the article reports a clinical trial, check the article for a trial registration number and search the relevant trial registry to determine whether the study was registered prospectively. The WHO expects that all clinical trials are prospectively registered in a WHO Registry Network approved registry. One of the minimum standards set out for trial registries in the International Standards for Clinical Trial Registries is that registries must obtain written third-party confirmation of a trial’s existence as part of the registration process. Lack of prospective registration can be one sign of a problematic study, however it should be noted that lack of evidence of prospective trial registration alone is not considered sufficient to exclude a study from a review, and there is no empirical evidence that inclusion of only prospectively registered studies in a review guarantees the inclusion of only trustworthy studies. For some types of study designs, such as qualitative research, prospective registration will not be possible.
If the article reports a clinical trial that was prospectively registered, check the list of outcomes in the trial registry entry against the outcomes that were reported in the study.
Check the article for a statement of whether approval from an ethics committee was granted for the study and whether this statement includes the name of the ethics committee granting the approval, and an approval reference number. Please note that lack of a statement regarding ethics approval does not mean that a study did not have ethics approval or that the study is untrustworthy and should not be included in a Cochrane Review.
Look for potential ‘red flags’ in the article, for example:
- Is the number of patients recruited within the timeframe with the condition plausible?
- Do the numbers of participants add up?
- Are there obvious inconsistencies in the article e.g. different country specified in different parts of the article?
- Are the authors affiliations all from a different country than the country the study is reported to have taken place in?
- Is there a realistic response rate or number of participants lost to follow up?
- Is the number of authors plausible for the study design? (e.g. is it a single author article reporting a randomised control trial)
- Is there significant overlap in text/data with another published articles by the same or different authors without explanation?
Check to see if the study has been included or excluded from any other Cochrane Reviews and, if so, if any additional information is available. Consult with an information specialist for help with this.
Consider further statistical assessment of the study data:
- An analysis of baseline data (e.g. age) can be used to identify unlikely baseline data, which may be the result of honest error, poor methodology or misconduct. For further information, see for example Carlisle 2015. Consult with a statistical editor for advice for advice about this.
- Sensitivity analyses can be used to prove that the findings from a systematic review are not dependent on arbitrary or unclear decisions and are therefore useful when considering whether one or more studies should be included in a meta-analysis. For further information, see the Cochrane Handbook, chapter 9.7 Sensitivity analyses.
Consider searching for studies authored by one or more authors of the study about which there are concerns, even if the studies were ineligible for inclusion in the current Cochrane Review, to look for retractions, Expressions of Concern and editorial comments. It is important to note that a retraction of another article by an author does not necessarily indicate that there is a problem with other studies on which they are an author.
Look for potential 'red flags' in individual participant data (if available) - see Dr Kyle Sheldrick's presentation "Seven signs of fraud in individual participant data", part of the NSW Health Statewide Biobank Virtual Seminar Series.
Use the research integrity assessment (RIA) tool to assess RCTs of investigational drugs in reviews using more recent studies. See Identifying and managing problematic trials: A research integrity assessment tool for randomized controlled trials in evidence synthesis

7.3 Examples of serious concerns about the trustworthiness of a study

The following cases give anonymised examples based on concerns that have been raised about studies that are eligible for inclusion in Cochrane Reviews.

Example 1

The authors of a review identified that a randomised controlled trial that was eligible for inclusion in a Cochrane Review that they were conducting contained overlapping text and data with a previously published article by different authors. The background, objectives, and conclusions of the articles contained identical text, but stated that they took place in different hospitals and between slightly different dates. The number of participants per group was identical between the two articles, as was much of the data reported. This raised concerns about the trustworthiness of the data reported in the second article.

Actions taken: The CRG Editor contacted the Editor of the journal that had published the second article raising their concerns and asking the journal to investigate. The article was placed the article in the “Awaiting Classification” section of the review.

Example 2

An Editor noticed that the distribution of the data in the control arm of a study that was eligible for inclusion in a Cochrane Review did not look plausible when compared to a more plausible looking distribution in another article. Further assessment showed that there were exactly equal numbers in each group, exactly equal mean at baseline between control and intervention, no negative changes, and no odd numbers). Based on this, they had serious concerns that the control arm changes were measured data.

Actions taken: The CRG Editor contacted the Editor of the journal that had published the study with a neutrally worded email detailing their specific concerns about the data in the control arm, including a description of the exact method they had used to determine that they had concerns about the data. The study was placed in the “Awaiting Classification” section of the review.

Example 3

While conducting a Cochrane Review the review authors noticed a study that was eligible for inclusion that had a very unlikely distribution of patients between the intervention and control groups. The review authors contacted the study authors for further information and they responded that “There was a simple randomization by tossing coin”. The review authors still had concerns after this explanation because the chance of the distribution of the intervention and control groups occurring was 0.000000000000033%. They therefore contacted the Editor of the journal that had published the study. The journal investigated and concluded that the article did not need to be retracted, but that the term “randomization” should be substituted in the article with "single-centre, prospective and observational study" as this reflected the methodological approach of the study. This review authors still had concerns about the study following this and therefore discussed it with the Research Integrity Editors. The study was excluded from the review as in this case the review included only RCTs and the reason for not including the study was stated neutrally in the review.

7.4 Template text for Editorial Notes on a Cochrane Review

Example text is provided below for Editorial Notes to alert readers of a Cochrane Review to a post publication amendment on an included study. The text for all Editorial Notes should be agreed with the Research Integrity Team and may need to be modified to reflect the exact circumstances. If you need to add an Editorial Note to a published review, you should contact the named Associate Editor who will refer to the Research Integrity Team.

Template text for when an included study has been retracted

“Expression of Concern: Readers are alerted that the included study X et al. 2016 has been retracted [citation for retraction notice]. The study will be moved to the Excluded Studies table and its impact on the review findings investigated and acted on accordingly.”.

Template text for when an included study has an Expression of Concern

“Expression of Concern: Readers are alerted that an Expression of Concern [citation for Expression of Concern, or original article if Expression of Concern is not citable] has been published on the included study X et al. 2016. The impact of the study on the review findings will be investigated and acted on accordingly.”

Page tree

Policy for managing potentially problematic studies: implementation guidance