Analysis graphs in DTA reviews

Before an analysis can be performed, you must have completed these steps:

To add an analysis:

  1. Go to the Analyses section in the left-hand navigation.
  2. Click Add analysis.
  3. Enter a name for the analysis.
  4. Select the tests to include.
  5. Select Subgroups if you want to examine accuracy by subgroups of studies.
  6. Select Paired data only if relevant. The Analyze Paired Data Only option can only be used when you have selected two tests (not more or less). This option allows you to restrict the analyses to those studies which have reported data on a pair of index tests.
  7. Select Confidence Interval (CI), Confidence region and Prediction region.
  8. After you have specified the name and basic properties of an analysis, you can manage settings for the forest plot and SROC plot in each separate tab.

Forest plot settings

A forest plot provides a visual overview of the results of individual studies in an analysis. It shows estimates of the sensitivity or specificity of individual studies together with their confidence intervals. It provides a visual representation of the amount of variation among results of different studies.

The following characteristics of a forest plot can be changed:

  • Covariates Displayed on Forest plot: to add information about certain covariates of individual studies to the forest plot.
  • Sort Study by: you can determine the ranking of individual studies within the forest plot. You can rank studies according to various factors, including the value of sensitivity or specificity, year of publication, methodological quality item or any other covariate you have specified.


The PDF submission file will include linked analyses at the end of the document, and linked analysis SVG files will be included in the production package.

As all analyses are automatically added at the end of the review publication, it is recommended not to include separate figures for any forest plots to avoid duplication, and to link directly to the analyses in the text. See Links in the text.

Analyses Properties: SROC plot settings

You can specify the characteristics of the SROC analysis.

  • Display SROC curves determine whether or not to draw an SROC curve for exploratory or the hierarchical (bivariate or HSROC) analysis (I.e., based on externally calculated parameters).
  • Axis Off means that the axes lines are removed, the scale markers are moved a little away from the graph boundaries and the background of the graph area is shaded light grey.
  • Display Study Points controls whether the pairs of sensitivity and specificity of individual studies will be displayed in the SROC plot.
  • Display CI on Study Points controls whether confidence intervals will be displayed as cross hairs on each study point in the SROC plot.
  • Symmetry determines whether to draw a SROC curve that is symmetrical around the diagonal line running from the upper left to lower right corner of the SROC plot (I.e. Constant odds ratio model) or asymmetric (I.e. Diagnostic odds ratio varies with threshold).
  • Use Weight for Analysis to specify the type of weight to be used in the Moses SROC analysis.
  • Use Scale for Size of Points to specify whether individual study points are displayed with equal size markers or markers where their sizes reflect differences in sample size, or inverse standard error. You can also have different markers according to the value of a covariate. If you have many overlapping points, you can use Percentage Scaling for All Points to reduce all point sizes by the same percentage.
  • Display Paired Data Lines controls whether lines are drawn between paired data points when Analyze Paired Data Only is selected. If enabled, you can also choose a line style and colour.


In DTA reviews, SROC curves must always be included as separate figures. See Add a figure.

Analyses Properties: Sources of heterogeneity

You can examine whether the accuracy of studies differ by certain characteristics of studies. A stratified analysis of studies will be performed showing whether accuracy of studies differ by subgroups of studies. No formal testing is possible in this version of RevMan to determine whether results differ significantly between subgroups. Such analyses need to be performed outside RevMan using statistical packages (see Additional analysis***). RevMan will show separate SROC curve for each subgroup on the SROC plot.

Subgroups of studies can be based on methodological quality items or based on any other specified covariate (using all, or only a subset of the covariate values).

To create an analysis showing subgroups in ROC space:

  1. Add a number of covariates using categorical data. See Covariates.
  2. Open the data table for the test. See Entering data.
  3. Assign the categories. There is a column for each covariate.
  4. Create a new analysis, set to use the Investigate Sources of Heterogeneity option.
  5. Open the Properties for the analysis and go to the Sources of Heterogeneity tab.
  6. Select the Covariate option, and use the pull-down list to choose a covariate.
  7. If you only want to use some of the covariate values, deselect the ones to exclude.
  8. Click Apply.

Graph colour, symbols and range

The hollow symbols represent individual study estimates, solid circles are the summary points (meta-analytical estimates of sensitivity and specificity), solid lines are the summary HSROC curves, dotted lines surrounding the summary points are #% confidence regions, and the dashed lines around these points are the #% prediction regions.

RevMan automatically chooses a colour and symbol to use for each test included in an analysis. You can also choose you own custom options for each test in the Analysis content panel.

You can use a custom Specificity Range to control the specificities over which the SROC curve will be drawn.

Additional analysis using results from outside RevMan

There is also the option to add the results of more complex models, like the HSROC and bivariate models (see the Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy). These hierarchical models can not be fitted in RevMan, but you can perform the analyses using a statistical package and then import the results into RevMan.


Externally calculated parameters

  1. After adding an analysis with subgroups, go to the Data tab.
  2. Select tests and subgroups.
  3. You need to choose a hierarchical model to perform meta-analysis of test accuracy studies.

There are two models:

a. HSROC model

Here you can enter the five parameters of the HSROC model:

    • Lambda – accuracy parameter.
    • Theta – cut-point parameter.
    • Beta– shape parameter.
    • Var(accuracy) – variance of accuracy parameter.
    • Var(threshold) – variance of threshold parameter.

These parameters can be calculated using BUGS or rjags software, or in SAS using Proc NLMIXED. SAS and rjags code for fitting the model is available in the appendix of Chapter 10 of the Handbook. Based on these parameters a SROC curve will be drawn.


b. Bivariate model

You can enter the five parameters of the bivariate model:

    • E(logitSe) – expected mean value of logit transformed sensitivity.
    • E(logitSp) – expected mean value of logit transformed specificity.
    • Var(logitSe) – between-study variance of logit transformed sensitivity.
    • Var(logitSp) – between study variance of logit transformed specificity.
    • One of the following:
      • Cov(logits) – covariance between logit transformed sensitivity and specificity, or
      • Corr(logits) –correlation between logit transformed of sensitivity and specificity

These parameters can be calculated using SAS, R or STATA software. Examples and code are available in the appendix of Chapter 10 of the Handbook.


References

Reitsma JB, Glas AS, Rutjes AWS, Scholten RJPM, Bossuyt PM, Zwinderman AH. Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews. Journal of Clinical Epidemiology. 2005;58(10):982-90.

Rutter CM, Gatsonis CA. A hierarchical regression approach to meta-analysis of diagnostic test accuracy evaluations. Stat Med. 2001;20(19):2865-84.

Macaskill P, Takwoingi Y, Deeks JJ, Gatsonis C. Chapter 9: Understanding meta-analysis. Draft version (4 October 2022) for inclusion in: Deeks JJ, Bossuyt PM, Leeflang MM, Takwoingi Y, editor(s). Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy Version 2. London: Cochrane.

Takwoingi Y, Dendukuri N, Schiller I, Rücker G, Jones HE, Partlett C, Macaskill P. Chapter 10: Undertaking meta-analysis. Draft version (4 October 2022) for inclusion in: Deeks JJ, Bossuyt PM, Leeflang MM, Takwoingi Y, editor(s). Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy Version 2. London: Cochrane.


Creating confidence and prediction regions

Based on the results of the bivariate model, you can also create a confidence region around the summary sensitivity and specificity as well as a prediction region (the region likely to contain the sensitivity and specificity of a new individual study). The following parameters need to be entered into RevMan:

  • SE(E(logitSe)) – standard error of the expected mean logit transformed sensitivity.
  • SE(E(logitSp)) – standard error of the expected mean logit transformed of specificity.
  • Cov(Es) – covariance between expected mean logit sensitivity and specificity.
  • Studies– the number of studies in the analysis.

Click one or more of the following check boxes on the SROC plot tab:

  • To obtain a summary curve, click check box in front of Display summary curve.
  • To obtain a summary point, click check box in front of Display summary point.
  • To obtain a confidence region (options are 90%, 95% or 99%, the default is 95%), click the checkbox in front of Display #% confidence region.
  • To obtain a prediction region (options are 50%, 90% or 95%, the default is 95%), click the checkbox in front of Display #% prediction region.