Facilitate use of Risk of Bias 2


Owner: Ella Flemyng
Roadmap history:  
Selected for Future in Q1 update (Top 3)
Selected for Future in Q3 update
Reason for archiving: Progress has already been made, remaining stories will be in the backlog

What business value does this roadmap item create? / Which problem is this roadmap item trying to solve?

Cochrane has rolled out RoB 2 as optional for review groups as part of a pilot process for its implementation. This tool needs to become mandatory for all new reviews end of 2020 (date TBC) and to do this, users are requesting further developments to RMW to optimise and facilitate the use of RoB 2 in RMW. Given that review groups are advised to complete the RoB 2 assessments in an Excel tool, an import from the tool is highly desirable to reduce manually entering all of the data in RMW, which is time-consuming, a duplication of efforts and increases the risk of inputting errors. The RoB 2 developers are in the process of creating an online version of the tool, which also needs to be integrated into RMW.

  • 49 review are enrolled in the Risk of Bias 2 pilot and another 16 that have expressed an interest
  • Two CRGs are in the scale-up stage, they will be encouraging all newly registered reviews to use RoB 2 going forward
  • A couple of new reviews every week join the pilot

Several author teams have reported that the process of transferring judgments is lengthy. 

How can the value delivered be measured?

To be confirmed:

  • Time saved in comparison to transferring data manually. 
  • Number of errors introduced to RoB 2 data through manually input (as well as less comprehensive supports for judgement for manual input)
  • Number of reviews opting-in to use RoB 2 by the end of 2020. 
  • Number of reviews published with RoB 2 by the end of 2020.
  • Interviews with pilot review teams (authors and editors) will help understand acceptability

Which assumptions are made in relation to the value?

The import of the judgments (the extra time it takes to use Risk of Bias 2 in comparison with Risk of Bias 1) is what blocks a large number of author teams to use Risk of Bias 2.

What are the dependencies for fulfilling the business value?

  • Developer of the excel tool to be available to set up the output of the excel tool that Cochrane requests.
  • Release date for publication of result table of reviews using Risk of Bias 2. 
  • The wider launch of Risk of Bias 2 is dependent on an excel import bein in place.

What are the risks related to this roadmap item?

What is included in the scope of this solution?

  • (MUST) Publish RoB 2 tables  RMW-1029 - Getting issue details... STATUS  (Available in the JATS view in Archie)
  • (MUST) Show Risk of bias result table in RevMan Web (Done - to be released with removal of the synchronization process ~Sep) 
  • (MUST, blocker for RoB 2 launch by the end of the year) Import judgments from Excel tool 
  • (SHOULD) Decide which analyses to show the RoB 2 result table for (to enable showing forest plots for sensitivity analyses but not the results table)
  • (SHOULD) Update the icons in the traffic plot to match the new RoB 2 table icons in a published review 
  • (SHOULD, only relevant when the online tool is available) Integration with online-tool
  • (COULD) Enable editing of Risk of Bias table

How much time do we estimate to implement this roadmap item?

This is a guesstimate made by the Review Production Team based on known information about the roadmap item and the capacity of the team at the time of estimation (June 2020).

Medium: 1-2 sprints = 2-4 weeks depending on what is included in the scope. Priorities between suggestions are clear so de-scoping should be straight-forward.