What business value does this roadmap item create? / Which problem is this roadmap item trying to solve?

Strategic fit

Link to strategy: GOAL 1 Streamlining production of reviews and simplifying editorial systems and processes

Future of review production project: New review format

Page describing JATS changes: New Review Format - JATS changes

Hypothesis

Moving content from JATS article to supplementary materials will

Reduce complexity of publication processing (incl. copy editing) and reduce costs for Wiley to generate Cochrane reviews for the Cochrane Library
Offer an opportunity to align Cochrane review article with standard journal formats and put us in a better position to
1. move to a different platform for Cochrane Library if needed
2. support Word-to-JATS conversion for direct submission
Make the review more acessible for readers as there will be less content in the article

Success metric

Not changing date after launch date is announced
Pre and post satistfaction of peer reviewers (New review format)
Author understanding of what is changed in the review after existing review has been converted to use supplementary materials (user interviews)
Pop-up to collect net promotor score
Number of support tickets related to supplementary materials and RevMan

Which assumptions are made in relation to the value?

The cost of publication processing can be significantly reduced by reducing the content of the JATS article and further standardizing it.

The user experience for writing a review can remain largely unaffected (i.e. reduction in user experience is not acceptable).

Quality measure?

What are the dependencies for fulfilling the business value?

Key decisions before we pick this item up to start development (or could be postponed by iteration):

Decide what content is in main article and what is supplementary materials
Agree packaging and XML changes (input from Wiley and Highwire required) (some minor XML changes still to be decided e.g. "Additional Tables" heading no longer makes sense)
- Design tier 1, 2, 3 formats for supplementary materials (Essentially done - any remaining changes likely to be updates to boilerplate text or styling, not structure of content (status of "search strategies" as inline display might change, but would not impact this roadmap item.)
- xdps and highwire work can progress based on samples
Decide how to host supplementary materials (input from Wiley and Highwire required)
Identifying and testing EM configurations for supplementary materials (In progress by Ursula Gonthier - I think we have a fairly good understanding already)
Agreed plan and timeline for tech implementation by Wiley and Highwire (In progress, I think Wiley are ready to run with it. For HighWire we will need the UX done first, I think.)
Agree roll-out strategy
Research and mockups to understand if and how RMW should change as supplementary materials are introduced
End-to-end testing will require RevMan work to be completed
Guidance & training will require RevMan Web development

Example files for review with supplementary materials:

CD015391-main.html (needs jats-preview.css one folder higher (..) in the hierarchy)
CD015391-main.xml
CD015391-SUP-01-searchStrategy.html
CD015391-SUP-02-characteristicsOfIncludedStudies.html
CD015391-SUP-03-characteristicsOfExcludedStudies.html
CD015391-SUP-04-characteristicsOfAwaitingStudies.html
CD015391-SUP-05-characteristicsOfOngoingStudies.html
CD015391-SUP-06-riskOfBias2.html
CD015391-SUP-07-analyses.html
CD015391-SUP-08-dataPackage.zip (references TBA)

Other examples:

CD012252.zip - overview
CD007146.zip - intervention; with "other" supplementary materials
CD009066.zip - intervention; with "other data table" analyses
CD009175.zip - DTA review

What are the risks related to this roadmap item?

Large project
- How can we iterate?
- Clarify what decisions are needed before we start development
- Mock whole workflow with data sets early on

What is included in the scope of this solution?

Must haves

Mockups for UI changes (left-hand navigation, internal link dialog, enable supplementary materials) to get user feedback
A way to enable supplementary materials on a review by review basis (TBD which permissions should allow you to do this author, super user or other)
- Need to work for reviews with content: when supplementary material feature is enabled
  - Cross-references to the characteristics of studies or references to studies sections are replaced by a cross-reference to the corresponding supplementary material
  - Cross-references to appendices are replaced by a cross-reference to the corresponding supplementary material
  - Cross-references to analyses and risk of bias 2 tables are converted to mentions (unlinked text)
  - Cross-references to studies will be converted to mentions (unlinked text); the first mention (other than in Summary of Findings tables) will include a cross-reference to all bibliographic references for the study
  - Links to bibliographic references in Other data type tables (main article), study characteristics tables and risk of bias tables are stripped
  - Do we do anything about round brackets around links to references and studies?
- All reviews using supplementary materials also has study-centric data and GRADEpro integration enabled? (this is likely to be addressed earlier when RevMan 5 is retired)
- What changes when you enable supplementary materials?
  - Content converted (as per above)
  - Indication on status panel
  - Left-hand navigation changing (unless we make the change for all reviews)
  - A new version is created
  - A version is saved before the new version is created
  - (TBD) Notifications in UI
  - Comments section no longer supported (need to store information elsewhere)
Search strategy is implemented as a supplement
- Search strategy is the first supplement available in the left hand menu
- You can add one search strategy (or should it exist from the start? Is there always a search strategy?)
- On convertion, search strategies in appendices are concatinated and included in the supplement Search strategy
- JATS v2 is updated to support search strategy as a supplement
Updates to internal link management
- In the JATS, links to references are numbered and ordered by where they were inserted first in the review
  - See Results user testing citations (part 2)
- While you are editing text, if you hover over a citation to a bibliographic reference or study the reference is shown and you can navigate to it (or a shortened version)
- Suggested: While you are editing text, if you hover over a link to a supplementary material we show the type or name (for appendices)
- When inserting links to bibliographic references, always insert square brackets around the reference (To be confirmed)
- TBD behavior when inserting study references
- When inserting links in supplementary materials, there are no "live" links to things outside the appendix (to the main article, external links work as before)
  - You cannot insert links to other references in any supplement except Search strategy or Other supplements
- When inserting links to studies or analyses in the Main article, only a reference to the study or analysis is inserted (no hyperlink) a.k.a mentions. While editing, you should still be able to navigate within RevMan (identify best way to indicate to user)
  - Don't show the reference for mentions, show study id or analysis name number with specific styling to indicate it has actions and you can navigate to it within RevMan but that it won't be hyperlinked in the article
  - We need to make sure people don't add citations inline where they become a part of the flow of the sentence (how can we do this?)
- Link options updated in internal link dialog:
  - There's a new category "Supplementary material"
    - The list contains all supplementary materials including each separate appendix and the Search strategy
    - The name of each supplement remains the same or should it be Supplementary material 1, supplementary material 2.. or do we need both?
  - Other sections changed to something more useful "(Headings in the main article?)" and only includes all fixed headings that are in the main article
  - Characteristics of included studies,exclued studies, ongoing studies are removed from Other sections
  - Risk of bias (tables) is removed from "Other section"
  - Data and analyses removed from Other section
  - Included studies, Excluded studies, Studies awaiting classification, Ongoing studies in other sections removed (assuming this refers to the reference lists)
  - Appendices removed from "Other section"
  - Previous versions of this review removed from "Other section" Why?
  - Additional references removed from "Other section"
  - Risk of bias (table) removed (you shouldn't be able to link to specific risk of bias tables)
  - Option to link to comments removed
  - In study characteristic tables, risk of bias tables (support for judgment) and other data tables, you cannot insert other (a.k.a bibliographic) references
Updates to left hand navigation (TBD pending UX workshop and user testing)
- You cannot navigate to top level sections Review information. Data, Content & supplementary materials
- Review information renamed to About this review (collapsed by default)
  - Authors
  - Dates and events
  - TBD if the following should be moved to article
    - Contributions of authors
    - Acknowledgements
  - TBD if this section should be removed to be included in the Methods: Differences between protocol and review
  - Published notes
- Data section introduced (expanded by default)
  - Review criteria
  - Studies
  - Analyses
- Contents section introduced (expanded by default)
  - Text renamed to Article
    - Includes SoF tables where they are situated in the article
  - References
  - Figures
  - Tables
    - No longer includes SoF tables
- Supplementary materials section introduced (collapsed by default?)
  - Search strategies (concatenated from search strategies
  - Study characteristics can be viewed, link to edit included studies
  - Risk of bias can be viewed, link to edit Analyses
  - Analyses (need to confirm name as duplicate with other section above)
  - Data package
    - Enable user to download data package that would be included as supplementary materials (user journey TBD)
  - Other (specific name to be confirmed: other supplementary materials, Other supplements, Other)
- Comments section in left hand navigation removed
Generate JATS for main article without using RevMan 5 library and RevMan 5 format & generate supplementary materials as htmlx
- RMW-3316 - Getting issue details... STATUS
- RMW-3317 - Getting issue details... STATUS
- RMW-3165 - Getting issue details... STATUS
- RMW-3304 - Getting issue details... STATUS
Validation changes
- For RoB 2: Validation error should be changed from checking whether at least one table is cross-referenced to checking that you have cross-referenced the supplementary material for Risk of bias
- Validation warning if you have links to specific risk of bias 2 tables
- Validation warning if you have hyperlinks in supplements that does not support it
An in application message to ask authors to send in interest in using supplementary materials (for reviews that does not yet have it enabled)
NICE TO HAVE: In application guide to show what is changed in supplementary materials
- Describe how mentions will work and how they should be used (where?)
- Describe that references will be
- Notify about what is moved to supplementary materials (New supplementary materials, left hand navigation)
- Describe changed data management process and link to guidance about study-centric data (New data section left hand navigation)
- Notify about changes to heading structure?
- Describe process for submitting?
Flag to authors which links need updating (for reviews with existing data)

Mockups: Supplementary materials.bmpr

Q: Is there any impact on validation rules?

Q: Will the "switch" be to enable new review format meaning future increments will be available as they are released and will not require active subscription? Some may need switches.

Out of scope

User uploaded supplementary files (but they can add any number of supplementary materials in RevMan Web)

How much time do we estimate to implement this epic?

Large (at least 6 sprints), noting that scoping work is excluded.

Creating JATS took months of development and years of testing including with Wiley, one element of this includes creating JATS for new review format.
Comparable with implementing support for DTA reviews (7 sprints)

This is a guesstimate made by the Review Production Team based on known information about the roadmap item and the capacity of the team at the time of estimation (2023-01-26).