Risk of Bias 2 in RevMan Web


Please start by reading the information on Risk of Bias 2 available from Cochrane Methods.

Reviews using Risk of Bias 2 (RoB 2) must be prepared in RevMan Web because the functionality to record and display RoB 2 will not be added to RevMan 5 (desktop version). RoB 2 use in a Cochrane Review must have approval from the relevant Cochrane Review Group. This approval must be obtained before the protocol has been drafted. Please ensure you discuss this with your Managing Editor before using RoB 2.

As a service to Cochrane authors, you can submit your completed risk of bias assessments to your Cochrane Review Group to check before you enter the RoB 2 assessments into RevMan Web, as this will facilitate peer review. For example, please submit the completed RoB 2 Excel tool.

Cochrane authors and Cochrane Review Group staff can also submit RoB 2 questions to the monthly Methods Support Unit Web Clinic for discussion – read more and submit questions here.

Important information regarding enabling Risk of Bias 2 feature

Enabling the RoB 2 tool in your review will break the compatibility between RevMan Web and RevMan 5.

  • When a review in RevMan Web is switched to ROB 2, you can no longer check the review out into RevMan 5.
  • It may therefore be advisable to wait for a specific point in the editorial process before enabling the integration (e.g. until after studies have been imported from Covidence as Covidence only imports to RevMan 5).
  • For review updates that have switched to RoB 2 from RoB 1, all risk of bias data from the previous version will be lost when RoB 2 is enabled. If authors need to download or make a copy, ensure they do it before RoB 2 is enabled. 
  • Once the RoB2 feature is enabled, it is not possible to disable it except to revert to a version of the review prior to enabling of the features. However, when reverting to a version prior to enabling RoB 2, RoB 1 data will not be recovered.
  • Authors can request that their Managing Editor enable this feature; more information about enabling RoB 2 available here.

RoB 2 tables and figures:

In most circumstances, authors do not need to create any RoB 2 tables or figures outside of RevMan. We recommend authors visualise RoB 2 data in a Cochrane Review by:

Both of these are created in RevMan Web.

1. Ensure you include the correct information in the support for judgement in RevMan Web.

If using the RoB 2 Excel tool (see the Tools section in the RoB 2 Cochrane Review Starter Pack – available via Cochrane methods), the following steps will ensure the support for judgements for each domain are correctly included in this published table and reduce the chance of delays during peer review:

Add justifications for all responses to the signalling questions in the ‘Description’ boxes and once complete for all signalling questions, double click on the box below to create the support for judgement for the risk of bias judgement for that domain.

Run the algorithm and then confirm your risk of bias judgement for that domain (assessor’s judgement). Edit the ‘support for judgement’ so that it is complete. This text can be copied into RevMan Web to populate the published table.

2. Switch to RoB 2.0

Ensure  your review is checked in and then email support@cochrane.org when you are ready to switch on RoB 2 functionality. We advise that this is after you have input your results data and have completed your risk of bias assessments using the tools developed to manage the process (see the Tools section in the RoB 2 Cochrane Review Starter Pack – available via Cochrane methods).

To check that your review is set up for using RoB 2.0, go to Included studies, then click on the ‘Risk of Bias Setup’ button.

3. Note the domains of the RoB 2.0 tool

The domains will now include the ones covered by the RoB 2.0 tool. Note that the first domain (Bias arising from the randomization process) is study-level, while all others are outcome-level. Also, note that all the domains are fixed and you cannot edit them, which is different from the RoB 1 tool.

4. Select the analysis

In the Analyses, go to the analysis (outcome) where you want to add your judgement, then click on 'Edit Analysis'.

If the outcome in this Analysis was pre-specified in the protocol for a Risk of Bias 2 assessment, ensure the ‘Show the risk of bias table’ box is checked in the ‘Options’ tab. This will create a results-level risk of bias table for this Analysis, which includes all domain judgements and support for judgements for this outcome (see Step 5 on ‘Set up an Analysis’).

5. Go to risk of bias judgements

To add a risk of bias judgement for a particular result, click on the 'Action' button on each study row and select 'Edit risk of bias'.

6. Enter the risk of bias judgements

For each domain, make a judgement (low risk of bias, some concerns, or high risk of bias) and add supporting text. This information populates the results-level, RoB 2 tables and therefore must be completed.
If the support for judgement text is not included this will raise a Validation Error in the Dashboard.

Please ensure you include the correct information in the support for judgement text box in RevMan Web.

At the moment this needs to be done manually, i.e. there is no way to automatically transfer these judgements from a file or an application.

As mentioned before, the first domain (Bias arising from the randomization process) is study-level and all changes made for this domain will apply to all results within the study.

7. View the risk of bias summary graph

To view the graph summary of your risk of bias judgements, go back to the analysis and click on the 'Graphs' tab. 

Tick the box in front of 'Show Risk of Bias summary' and then the risk of bias traffic lights will be added to the forest plot in the associated Analysis section of the published review. There is no need to create figures from an Analysis as you can reference the relevant Analysis within the review text.

The results for which you have not entered the risk of bias judgements are indicated by orange circles with exclamation marks.

8. View the risk of bias table

Please note that you cannot currently add footnotes to a Risk of Bias 2 table in RevMan Web.

Each outcome prespecified for RoB 2 assessments (likely to be the reviews’ critical and important outcomes included in the SoF table) should have a table that includes the risk of bias judgements (high, low or some concerns) and the supporting text for each judgement. 

Once you have created the main Analysis, added the risk of bias assessments and ticked 'show risk of bias table' in the analysis options, you can view a version to see the risk of bias table. 

In the generated view, scroll down to the ‘Risk of bias’ section to view the results-level RoB 2 tables, which are located after the characteristics of studies section.

9. Duplicating the Analysis for a sensitivity or subgroup analyses that include the risk of bias assessments 

Once you have created the main Analysis and added the risk of bias assessments, you can duplicate this for the sensitivity or subgroup analysis. This prevents you from needing to re-input the data and risk of bias assessments. To duplicate, select the Analysis and go to ‘Actions’. Select ‘Duplicate Analysis’ and select the destination analysis group.

Please note that it is not possible to store two sets of ROB 2 assessments for a single study, if for instance a study with two subgroups has different ROB 2 assessments for each of the subgroups. There will be only one set of ROB 2 judgements available per subgroup.

See this help video.

The duplicated Analysis will be added below your main Analysis.

Select the new Analysis. You can change the name on the Analysis in the ‘Options’ tab and remove the studies that you don’t want included in that sensitivity analysis in the ‘Data’ tab by selecting ‘Delete data row’ under ‘Actions’.

There is more information in this Knowledge Base onsensitivity analysis’ and ‘subgroup analysis’.

10. Deleting a Risk of Bias assessment

If there is an analysis or study for which you would like to remove the Risk of Bias assessment, there are two ways to do this. 

If you do not want a Risk of Bias assessment to publish for a full Analysis, you can uncheck the "Show risk of bias table" for that Analysis.

If you want to delete the risk of bias assessment for a particular study (e.g., if you copied created the analysis by duplicating another analysis for which Risk of Bias was already determined, but it is not relevant for that analysis), you can select the study, click on the Action menu, and select Delete risk of bias.