RETIRED. See the Editorial and Publishing architecture-overview.docx for how it has been replaced.

Delivery of When Ready content to Wiley and Cochrane.org
Rasmus Moustgaard

Versions of this document

  • 12 August 2022: Make document reflect current state of the API and how it's used
  • 2 December 2021: Draft update for removal of review publishing via Archie.
  • 15 January 2021: Draft update for production improvements: publication date determined by HighWire & Aries JATS packaging (Gert van Valkenhoef)
  • 2 July 2019: Draft update for new review publishing file format ("JATS") project (Gert van Valkenhoef).
  • 12 July 2016: Updated DTD. Added missing licenceType parameter to description of Test Method.
  • 13 November 2015: Minor updates for consistency + added aim of document (Rasmus Moustgaard + Ida Wedel-Heinen)
  • 03 November 2015: Added section about Review Updating Classification Framework, added the metadata file DTD as Appendix 1, and minor updates for consistency ()
  • 11 January 2013: Added information under 'Test Method' (Rasmus Moustgaard)
  • 2012: First version describing the "When Ready" publishing method under development (Rasmus Moustgaard)

Aim of document

To describe workflow, services and data relating to publication of Cochrane reviews using the "When Ready" publishing method via Archie, and notification to Archie of reviews published via Aries Production Manager.

Workflow - content published via Archie

As of January 2022, only translations and topics lists are published via this Archie API.

  1. When translations are marked for publication in Archie, Archie sets the datePublicationMarked field in its database.
  2. Wiley will access a web service method (Get Content for Publication) to fetch a data package (zip archive, see Data format) with all the content from Archie that is ready for publication. Archie will deliver all content that has not yet been delivered and has the requested delivery date (datePublicationMarked field) set before or equal to the current date/time.
    1. The global browse menu (EDITORIAL topics list) is included in the zip archive once daily (at 2 PM UTC), being a snapshot of the latest data in Archie at the time of delivery (without explicit "mark for publication").
    2. Wiley can access the Get Content for Publication method at any time to fetch the latest content; however, 15-minute intervals are expected according to specifications.
    3. The web service will never return multiple versions of the same content in the same call.
  3. On receipt and basic checking of content, Wiley will make a call-back (Content Received) to confirm that the content has been received. Wiley will send the list of received content as an XML document in a custom format (see below) where content is identified using a combination of ID and Archie version number. At this point, Archie will record the actual delivery date/time (datePublicationDelivered field) of the content that was successfully delivered to Wiley.
    1. Note: These callbacks are also expected to occur for content published via ProduXion Manager (see below)
  4. When the delivered content is actually published (as opposed to being picked up), Wiley will make a second web service call-back to Archie (Content Published), this time providing the list of content that was published. At this point Archie will set the publication date (datePublished field) to the current date/time (and also make other changes to the database like setting the published flag for the published versions and marking them as the latest published versions).
    1. Note: These callbacks are also expected to occur for content published via ProduXion Manager (see below)

Workflow - content published via ProduXion Manager

See Aries ProduXion Manager publishing workflow for a description of how this workflow works in ProduXion Manager.

  1. When a review is ready for publication, ProduXion Manager delivers the final JATS package to Archie via SFTP. Archie archives the JATS and displays the DOI as "withheld from delivery". The content will not be available via the web service (Get Content for Publication).
  2. Following delivery to Archie, the JATS package will be delivered to Wiley via SFTP. Wiley will acknowledge receipt via the Content Received callback to Archie.
  3. Following final publication, Wiley will acknowledge this via the Content Published callback to Archie.

Workflow - third party delivery of published content

Once content has been published (whether via Archie or via ProduXion Manager) it is available for publication on Cochrane.org (and potentially to the CRS or other clients). They access another Archie web service (Get Published Content) to fetch all the reviews and translations that were published during a given period, specified by a start date/time and an end date/time. Archie delivers all the reviews and translations with a publication date (datePublished field) between the specified dates. The client is responsible for keeping track of start dates and end dates to ensure that no period is queried twice.

The Get Published Content service may be called at any interval decided by the client. The format of the data package will be the same as for the Get Content for Publication service. As it is possible that notification of publication to Archie is delayed, a minimum delay of 15 minutes in retrieving published content is recommended. Notifications that are delayed by more than 15 minutes are monitored and will be communicated to users of this API via email.

Data package format

A data package will be delivered as a zip archive. The file/folder structure inside the archive will match the structure of this example (with only Get Published Content including reviews):

  • AIRWAYS
    • reviews
      • CD005304.pub3.zip
      • CD008250.pub2.zip
      • CD010005.pub2.zip
      • CD010126.zip
      • CD010139.zip
      • CD010177.zip
      • CD010178.zip
    • translations
      • CD001116.pub3.fr.zip
      • CD006100.pub2.fr.zip
      • CD008250.pub2.fr.zip
      • CD008469.pub2.fr.zip
  • ANAESTH
    • reviews
      • CD003587.pub2.zip
      • CD007105 .pub2.zip
      • CD010134.zip
      • CD010135.zip
      • CD010160.zip
  • ARI
    • reviews
      • CD004882 .pub3.zip
      • CD006822 .pub3.zip
      • CD008116 .pub2.zip
      • CD008268 .pub2.zip
      • CD008723 .pub2.zip
      • CD008858 .pub2.zip
      • CD009088 .pub2.zip
      • CD010130.zip
    • translations
      • CD004882.pub3.fr.zip
      • CD006206.pub3.fr.zip
  • BACK
    • translations
      • CD004250.pub4.fr.zip
      • CD001351.pub2.fr.retracted
  • EDITORIAL
    • topics.xml
  • output.log
  • success.log

Notes to the package structure:

  • In order to demonstrate as many aspects as possible, this example has been made unrealistically large. With 15 minute pickup intervals the majority of packages will be empty or only contain a single review.
  • The root folders represent Cochrane groups (review groups and the EDITORIAL group) and are named with the entities' publisher IDs. Root folders will only be present for entities with content ready to publish.
  • The reviews sub-folder contains published reviews (only for "Get Published Content"). The structure of individual ZIP files is specified elsewhere: JATS - changes for Aries compatibility. The folder will be absent if no reviews are included in the package.

  • The file name of each individual review follows this convention:
    [CD number].[pub number].zip
    NOTE: .[pub number] is absent for first publication (i.e. ".pub1" is omitted).

  • The translations sub-folder contains translations of reviews, but not necessarily for the included reviews. The folder will be absent if no translations are included.

  • The file name of each individual translation follows this convention:
    [CD number of review].[pub number of review].[language code].zip
    For example: CD001478.pub4.da.xml

  • The EDITORIAL/topics.xml file contains the browse menu (topics).
  • Notification about a retracted translation is handled by including the latest version of the translation with a filename according to the following convention:
    [CD number of review].[pub number of review].[language code].retracted
    For example: CD001351.pub2.fr.retracted
  • The root file output.log contains log entries from the export process. In addition, either an empty success.log file or an error.log file (with details of the error) will be present.

Service specifications

Interfaces

Services are accessible as SOAP based web service calls.
WSDL URL: https://archie.cochrane.org/jboss-net/services/PublishingService?WSDL
WSDL URL for testing: https://test.archie.cochrane.org/jboss-net/services/PublishingService?WSDL
Target namespace = https://archie.cochrane.org/service
Service name = "PublishingService"

Security

A stateless request/response model is adopted for communication between Archie and Wiley/Cochrane.org. Each request is authenticated using a password/token agreed between the parties. SSL/TLS is available for encryption of communication.

Service methods

Get Content for Publication

Retrieves content ready for publication that has not yet been recorded as received.

  • Method: byte[] PublishingService.getContentForPublication(String token)
  • Method parameters:
    • token: Authentication code/password
  • Returned value: A zip archive with content ready for publication in the format described under Data package format.
  • Error handling: See below.

Content Received

Records that content has been received. Should be called before making further calls to Get Content for Publication.

  • Method: byte[] PublishingService.contentReceived(String token, String receivedContent)
  • Method parameters:
    • token: Authentication code/password.
    • receivedContent: XML snippet listing the content that was received by Wiley.
  • Returned value: A zip archive will be returned for consistency with other service methods. The zip archive will contain an empty success.log file in case of no error.
  • Error handling: See below.

XML format for received content:

<content success="yes|no" error="">
  <review cd_number="CD005304" version_no="4.0" success="yes|no" error=""/>
  <translation id="z1209211141410854187737993820866" version_no="0.0" success="yes|no" error=""/>
  ...
</content>

One element should be included for each review and translation. If received content contains an error that prevents further processing the success attribute should be set to 'no' and details may be provided in the error attribute. If it is not possible to process the received zip archive all, success="no" should be specified on the content root element. In this case the ITS Team will be notified automatically.

Content Published

Records that content has been published.

  • Method: byte[] PublishingService.contentPublished(String token, String publishedContent)
  • Method parameters:
    • token: Authentication code/password
    • publishedContent: XML snippet listing the content that was published by Wiley. For reviews, this includes the actual published date and time.
  • Returned value: A zip archive will be returned for consistency with other service methods. The zip archive will contain an empty success.log file in case of no error.
  • Error handling: See below.

XML format for received content:

<content success="yes|no" error="">
  <review cd_number="CD005304" version_no="4.0" success="yes|no" error="" published="2007-04-05T14:30:07.243+01:00" />
  <translation id="z1209211141410854187737993820866" version_no="0.0" success="yes|no" error=""/>
  ...
</content>

One element should be included for each review and translation. If received content contains an error that prevents publication the success attribute should be set to 'no' and details may be provided in the error attribute. The published timestamp must have at least precision to the closest second, and must specify the timezone.

Get Received Content

Retrieves content again that has already been recorded as received but has not yet been recorded as published. For testing and emergency use.

  • Method: byte[] PublishingService.getReceivedContent (String token)
  • Method parameters:
    • token: Authentication code/password
  • Returned value: A zip archive with content that has already been recorded as received in the format described under Data package format.
  • Error handling: See below.

Get Published Content

Retrieves content that has been recorded as published. For Cochrane.org.

  • Method: byte[] PublishingService.getPublishedContent(String token, Date startDate, Date endDate)
  • Method parameters:
    • token: Authentication code/password
    • startDate: Start date/time used for filtering returned content
    • endDate: End date/time used for filtering returned content
  • Returned value: A zip archive with content published between startDate and endDate in the format described under Data package format.
  • Error handling: See below.

Test Method

Marks a random sample of reviews and translations for publication. Note, currently this service does not generate Updating Classifications for the marked reviews.

  • Method: byte[] markTestSampleForPublication(String token, int nReviews, int nTranslations, Date earliestDeliveryDate, int nDeletedProtocols)
  • Method parameters:
    • token: Authentication code/password.
    • nReviews: Number of reviews to mark for publication.
    • nTranslations: Number of translations to mark for publication.
    • earliestDeliveryDate: Start date/time for delivery by getContentForPublication.
    • nDeletedProtocols: Number of protocols to inactivate and mark for publication.
    • licenceType: Type of licence to associate with the published reviews. 0=DEFAULT (Green open access), 1=CC_BY (Attribution), 2=CC_BY_NC (Attribution, non-commercial), 3=CC_BY_NC_ND (Attribution, non-commercial, no derivatives).

Error handling

In case of success the returned zip archive will contain an empty success.log file in the root folder.
In case of an uncontrolled error, e.g. a network problem or the Archie server not responding, the results are unpredictable. It is recommended to try again a reasonable number of times, and then contact the ITS Team. Depending on the type of error the ITS Team may have been notified automatically.
In case of a 'controlled' error, the zip archive will contain an error.log file with details. For instance, if the error is due to incorrect password or invalid parameters this will show from the error.log file (details to be finalised). Again, a reasonably number of retries should be attempted before contacting the ITS Team. Depending on the type of error the ITS Team may have been notified automatically.
Content received from an unsuccessful call should generally be discarded and not published. Any controlled error will cause the database in Archie to roll back to the state before the call was made.

Potential error conditions

  • Uncontrolled errors
    • No network connection between Wiley and Archie
    • Archie server is not running/responding
    • Archie is not running/responding
    • Archie is running but web service is not available, e.g. no WSDL access
    • Web service method call returns a runtime error (bug)
    • Web service returns another response than a zip archive (bug)
    • Web service returns an invalid zip archive
  • Controlled errors (web service returns a zip archive with an error.log file)
    • Error.log file reports a runtime error (bug)
    • Error.log file reports a locking exception (temporary)
    • Error.log file reports another problem collecting data, e.g. because of invalid data
    • Error.log file reports that the method has already been called
    • Error.log file reports invalid formatted parameters to method call
    • Error.log file reports invalid values of parameters to method call
    • Error.log file reports authentication error
  • No labels