Versions Compared

Old Version 3

changes.mady.by.user Justin Mann

Saved on

compared with

New Version Current

changes.mady.by.user Ella Flemyng

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

The retracted study should be listed in the “excluded articles” studies” section, with the reason noted, and including a citation to the published retraction notice. This reason must be noted using neutral language and should not accuse authors of scientific misconduct. If there is any doubt about the phrasing of such a note, the CRG must contact the named Associate Editor and Senior Editor for the CRG Network (who will refer on to the Deputy Editor in ChiefResearch Integrity Team) before publication of the Review.

...

If an included study is retracted after the Cochrane Review has been published, the study data must be removed from all analyses and the extent of changes required in response to the retraction should be determined. While this takes place, an Editorial Note should be added to the Cochrane Review to clearly alert the reader that an included study has been retracted. The wording of the Editorial Note should be agreed with the EMD before publication and should cite the retraction notice. The CRG should contact the Network named Associate Editor (who will refer on to the Research Integrity Team as needed). See section 7.4 Template text for Editorial Note on a Cochrane Review for template text.

...

Depending on the reason for the published Expression of Concern, it may be appropriate to exclude the study or its data from the Cochrane Review (e.g. if the Expression of Concern is for serious concerns about the validity of the data). Note that Expressions of Concern may be published for reasons that do not affect the validity of the data (e.g. authorship disputes), in which case it may be considered appropriate to include the study in the Cochrane Review. If uncertain, contact the Network named Associate Editor (who will refer on to the Research Integrity Team as needed).

...

If the Expression of Concern relates to the validity of the data in the included study, an Editorial Note should be added to the Cochrane Review to acknowledge that concerns have been raised about one or more studies and/or study data included in the Cochrane Review. This should remain in place until either the Expression of Concern on the included study or the included study itself is retracted. The text of the Editorial Note on the review should be agreed with the Network named Associate Editor, who will refer to the Research Integrity Team, and should cite the Expression of Concern that is published on the included study. See section 7.4 Template text for Editorial Note on a Cochrane Review for template text.

...

  • Check that there are no post publication amendments published about the study. Search for the study in PubMed, Medline (and other bibliographic databases, as appropriate), the Retraction Watch Database, and PubPeer to clarify if there are any Expressions of Concern, comments, or Letters to the Editor regarding the study, and to confirm that the study has not been retracted. See section 1 Searching for post-publication amendments and consult with an Information Specialist if further assistance is needed with how to do this.
  • If an Expression of Concern is identified, the actions outlined in section 2.2 What to do when there is a published Expression of Concern should be followed.
  • If the study has been retracted, the actions outlined in section 2.1 What to do when a study is retracted should be followed.
  • Check the version of the article on the publisher’s website, for any un-indexed notifications, e.g. ‘Editorial Note’ or links to post-publication amendments that do not (yet) appear on mirror sites/bibliographic databases.
  • Describe your concerns in detail, including the exact method(s) used to determine that there may be a problem with the study, in an email to the Journal Editor using neutral language. You may find that the act of drafting this email crystalizes whether you have significant concern about the study if you are unsure. All language used must be kept neutral and follow the templates provided in section 7.1 Templates for corresponding with Journal Editors and authors. No accusations of scientific misconduct must be made. You should inform the Network Associate Editor, who will refer to the Research Integrity Team, before contacting the journal editor. This allows the Research Integrity team to keep a track of such cases which will inform updates to this guidance. It is the responsibility of the Journal Editor to initiate further investigation (for example contacting the Author’s institution).
  • If you are unsure whether you have sufficient concerns about an included study to warrant contacting the Journal Editor, you may wish to consider following some of the steps described in section 7.2 Methods for determining whether you have concerns about a study. Please note however that these methods are all unvalidated, and do not necessarily indicate that a study is untrustworthy. They may, however, help to consolidate, strengthen, or alleviate your concerns about a particular study.

2.3.1 Resolution for an unpublished review (concerns about an included study)

...

  • If information is missing from the report of the included study and/or further information is required to determine whether you have concerns, contact the authors of the study to ask for clarification and/or unpublished information in line with MECIR standard C49. All correspondence must be kept neutral. Do not accuse authors of misconduct, or fabrication or falsification of data.
  • Describe your concerns in detail, including the exact method(s) used to determine that there may be a problem with the study, in an email to the Journal Editor using neutral language. You may find that the act of drafting this email crystalizes whether you have significant concern about the study if you are unsure. All language used must be kept neutral and follow the templates provided in section 7.1 Templates for corresponding with Journal Editors and authors. No accusations of scientific misconduct must be made. You should inform the named Associate Editor, who will refer to the Research Integrity Team, before contacting the journal editor. This allows the Research Integrity team to keep a track of such cases which will inform updates to this guidance. It is the responsibility of the Journal Editor to initiate further investigation (for example contacting the Author’s institution).
  • If you are unsure whether you have sufficient concerns about an included study to warrant contacting the Journal Editor, you may wish to consider following some of the steps described in section 7.2 Methods for determining whether you have concerns about a study. Please note however that these methods are all unvalidated, and do not necessarily indicate that a study is untrustworthy. They may, however, help to consolidate, strengthen, or alleviate your concerns about a particular study.

2.3.1 Resolution for an unpublished review (concerns about an included study)

If, after attempting to draft a letter to the Journal Editor as described in section 7.2 Methods for determining whether you have concerns about a study, you decide that you do not have sufficient concerns about the study to contact the Journal Editor, the study , you decide that you do not have sufficient concerns about the study to contact the Journal Editor, the study can be included in the Cochrane Review. No further action is needed.

If concerns remain and you contact the Journal Editor to request an investigation and/or clarification of whether the data can be included in a Cochrane Review (which can take a considerable time where a Journal and potentially also an institution is involved) the publication of a new or updated Cochrane Review should not usually be delayed. If you have sufficient concerns about an included study to warrant contacting the Journal Editor, the study of concern should be categorized as “Awaiting Classification”, with a note added to explain why. The language of the note must be kept neutral, for example “The data could not be verified”. If any other note of explanation is required, you must contact the Network named Associate Editor, who will refer to the Research Integrity Team.

...

If concerns remain and you contact the Journal Editor to request an investigation and/or clarification to determine whether the data can be included in a Cochrane Review (which can take a considerable time where a Journal and potentially also an institution is involved), an Editorial Note should be added to the Cochrane Review to state that the journal editor has been contacted with concerns about the included study. The statement should remain as neutral as possible and include links to any published articles that might provide additional information (e.g. Letter to the Editor, etc.) and propose a timeline for any update (if known). The text of any Editorial Note must be agreed with the Research Integrity Team before publication. You should contact the Network named Associate Editor who will refer to the Research Integrity Team.

...

If the Journal Editor does not respond when concerns are raised, and no response is received on following up with the Journal Editor, consider contacting the Journal Publisher. If no response is received from the Journal Publisher, contact the Network named Associate Editor who will refer to the Research Integrity Editor for advice.

...

If a Correction article is published on the included study, the effect of this on the Cochrane Review should be determined and the Review updated if the correction affects the outcome of the review. If an Editorial Note was added to the Cochrane Review, this will need to be updated as appropriate. The Network named Associate Editor, who will refer on to the Research Integrity Team, should be contacted to agree the wording.

...

Support and advice are available to CRGs from the EMD on managing cases where there are concerns about the trustworthiness of an included study, or a study that is eligible for inclusion in a Cochrane Review. You can access this support by contacting the Network named Associate Editor who will refer to the Research Integrity Team as needed. They will advise on the wording of communications to authors, journal editors, institutions and also text within a Cochrane Review (for example, explaining why data from a study were excluded from the analyses, or why a study is “awaiting classification”) to mitigate the risk of defamation/libel. If further legal advice or support is required, it can be arranged by the EMD.

...

All correspondence must be kept neutral and the following templates should be followed. If a template is deemed inappropriate for a particular situation or the CRG would like to deviate from the template, the Network named Associate Editor must be contacted, and will refer to the Research Integrity Team, before contacting the study author(s) or Journal Editor. Correspondence must never accuse individuals of misconduct or fraud.

...

  • Consider assessing the study against an (unvalidated) checklist such as the Trustworthiness Screening Tool (TST) developed by the Cochrane Pregnancy and Childbirth Group or the REAPPRAISED checklist for evaluation of publication integrity.
  • If the article reports a clinical trial, check the article for a trial registration number and search the relevant trial registry to determine whether the study was registered prospectively. The WHO expects that all clinical trials are prospectively registered in a WHO Registry Network approved registry. One of the minimum standards set out for trial registries in the International Standards for Clinical Trial Registries Trial is that registries must obtain written third-party confirmation of a trial’s existence as part of the registration process. Lack of prospective registration can be one sign of a problematic study, however it should be noted that lack of evidence of prospective trial registration alone is not considered sufficient to exclude a study from a review, and there is no empirical evidence that inclusion of only prospectively registered studies in a review guarantees the inclusion of only trustworthy studies. For some types of study designs, such as qualitative research, prospective registration will not be possible.
  • If the article reports a clinical trial that was prospectively registered, check the list of outcomes in the trial registry entry against the outcomes that were reported in the study.
  • Check the article for a statement of whether approval from an ethics committee was granted for the study and whether this statement includes the name of the ethics committee granting the approval, and an approval reference number. Please note that lack of a statement regarding ethics approval does not mean that a study did not have ethics approval or that the study is untrustworthy and should not be included in a Cochrane Review.
  • Look for potential ‘red flags’ in the article, for example:
    • Is the number of patients recruited within the timeframe with the condition plausible?
    • Do the numbers of participants add up?
    • Are there obvious inconsistencies in the article e.g. different country specified in different parts of the article?
    • Are the authors affiliations all from a different country than the country the study is reported to have taken place in?
    • Is there a realistic response rate or number of participants lost to follow up?
    • Is the number of authors plausible for the study design? (e.g. is it a single author article reporting a randomised control trial)
    • Is there significant overlap in text/data with another published articles by the same or different authors without explanation?
  • Check to see if the study has been included or excluded from any other Cochrane Reviews and, if so, if any additional information is available. Consult with an information specialist for help with this.
  • Consider further statistical assessment of the study data:
    • An analysis of baseline data (e.g. age) can be used to identify unlikely baseline data, which may be the result of honest error, poor methodology or misconduct. For further information, see for example Carlisle 2015. Consult with a statistical editor for advice for advice about this.
    • Sensitivity analyses can be used to prove that the findings from a systematic review are not dependent on arbitrary or unclear decisions and are therefore useful when considering whether one or more studies should be included in a meta-analysis. For further information, see the Cochrane Handbook, chapter 9.7 Sensitivity analyses.
  • Consider searching for studies authored by one or more authors of the study about which there are concerns, even if the studies were ineligible for inclusion in the current Cochrane Review, to look for retractions, Expressions of Concern and editorial comments. It is important to note that a retraction of another article by an author does not necessarily indicate that there is a problem with other studies on which they are an author.

7.3 Examples of serious concerns about the trustworthiness of a study

7.3 Examples of serious concerns about the trustworthiness of a study

The following cases give The following cases give anonymised examples based on concerns that have been raised about studies that are eligible for inclusion in Cochrane Reviews.

...

The authors of a review identified that a randomised controlled trial that was eligible for inclusion in a Cochrane Review that they were conducting contained overlapping text and data with a previously published article by different authors. The background, objectives, and conclusions of the articles contained identical text, but stated that they took place in different hospitals and between slightly different dates. The number of participants per group was identical between the two articles, as was much of the data reported. This raised concerns about the trustworthiness of the data reported in the second articlemuch of the data reported. This raised concerns about the trustworthiness of the data reported in the second article.

Actions taken: The CRG Editor contacted the Editor of the journal that had published the second article raising their concerns and asking the journal to investigate. The article was placed the article in the “Awaiting Classification” section of the review.

Example 2

An Editor noticed that the distribution of the data in the control arm of a study that was eligible for inclusion in a Cochrane Review did not look plausible when compared to a more plausible looking distribution in another article. Further assessment showed that there were exactly equal numbers in each group, exactly equal mean at baseline between control and intervention, no negative changes, and no odd numbers). Based on this, they had serious concerns that the control arm changes were measured data.

Actions taken: The CRG Editor contacted the Editor of the journal that had published the second article raising their concerns and asking the journal to investigate. The article was placed the article study with a neutrally worded email detailing their specific concerns about the data in the control arm, including a description of the exact method they had used to determine that they had concerns about the data. The study was placed in the “Awaiting Classification” section of the review.

Example 23

While conducting a Cochrane Review the review authors noticed An Editor noticed that the distribution of the data in the control arm of a study that was eligible for inclusion in a Cochrane Review did not look plausible when compared to a more plausible looking distribution in another article. Further assessment showed that there were exactly equal numbers in each group, exactly equal mean at baseline between control and intervention, no negative changes, and no odd numbers). Based on this, they had serious concerns that the control arm changes were measured data.Actions taken: The CRG Editor that had a very unlikely distribution of patients between the intervention and control groups. The review authors contacted the study authors for further information and they responded that “There was a simple randomization by tossing coin”. The review authors still had concerns after this explanation because the chance of the distribution of the intervention and control groups occurring was 0.000000000000033%. They therefore contacted the Editor of the journal that had published the study with a neutrally worded email detailing their specific concerns about the data in the control arm, including a description of the exact method they had used to determine that they had concerns about the data. The study was placed in the “Awaiting Classification” section of the review.. The journal investigated and concluded that the article did not need to be retracted, but that the term “randomization” should be substituted in the article with "single-centre, prospective and observational study" as this reflected the methodological approach of the study. This review authors still had concerns about the study following this and therefore discussed it with the Research Integrity Editors. The study was excluded from the review as in this case the review included only RCTs and the reason for not including the study was stated neutrally in the review. 

7.4 Template text for Editorial Notes on a Cochrane Review

Example text is provided below for Editorial Notes to alert readers of a Cochrane Review to a post publication amendment on an included study. The text for all Editorial Notes should be agreed with the Research Integrity Team and may need to be modified to reflect the exact circumstances. If you need to add an Editorial Note to a published review, you should contact the Network named Associate Editor who will refer to the Research Integrity Team.

...