If not provided in the article, the contingency table of true positives, false negatives, true negatives and false positives was constructed for each cut-off point assessed based on the available information, usually sensitivity, specificity and prevalence of the disorder according to the gold standard.
The suggested cut-point for the scale is 16, but many studies presented accuracy results of the CES-D using other cut-points, since they aimed to evaluate and compare their performance and select the optimal value in that specific population. However, for the assessment of overall performance of the scale using information from all the 28 studies, we selected results for only one cut-off point per study, so that each study contributes only one estimate of sensitivity and specificity as required by the statistical methods applied[21 ]. We chose the cut-off point of 16 whenever possible, as this is the value usually recommended for the detection of depression with the CES-D. Notwithstanding, when a study did not report diagnostic accuracy results for cut-off 16, we used the cut-off point reported in that particular study. When more than one cut-off point was reported in an article, and in order to avoid multiple testing effects, we selected the cut-off with the best diagnostic accuracy within the study. We obtained the coupled forest plot reporting the raw data consisting of the 2x2 sensitivity and specificity table from each study, as well as the estimated sensitivity (SN; the proportion of true cases correctly classified by the cut-off point) and specificity (SP; the proportion of true non-cases correctly classified) for detection of depression of each of the studies, together with 95% confidence intervals. In the context of meta-analysis, when a variety of sensitivity and specificity values for a given test are available from several independent studies depending on the cut-off point, the summary receiver operating characteristic (SROC) has been proposed as a way to assess diagnostic data [25 (link)]. The SROC curve considers both sensitivity and specificity and the relationship between them, taking into account that not all studies used the same cut-off. It is assumed that different values of sensitivity and specificity apply if the cut-off point defining a positive test result varies from study to study, everything else being equal. Several procedures have been proposed to estimate the SROC curve from a set of independent studies [25 (link)–27 (link)]. Here, the Rutter and Gatsonis mixed effects model [26 (link)] was fitted to estimate the SROC curve, and the sensitivity and specificity of each study, weighted by study size were plotted in the ROC space. The area under the curve (AUC) for the fitted SROC was computed from the estimated diagnostic odds ratio (DOR) following the method described by Walter [28 (link)]. Also, for the subsample of studies that provided diagnostic accuracy results for the cut-off point 16 (n = 22), we estimated a bivariate meta-regression [27 (link)], which allowed us to obtain pooled estimates of a range of summary performance measures of the test’s ability to detect the presence of a disease for a given cut-off point. Specifically, the summary measures obtained were: a) specificity and sensitivity, and their corresponding 95% confidence intervals; b) the positive likelihood ratio (LR+) that described how many times more likely positive test results were in the diseased group compared to the non-diseased group; c) the negative likelihood ratio (LR-), describing how many times less likely negative test results were in the diseased group compared to the non-diseased group; and d) the DOR, that summarizes the diagnostic accuracy of the test as a single number describing how many times higher the odds are of obtaining a positive test result in a diseased rather than in a non-diseased person. Additionally, we evaluated the screening accuracy of other cut-off points that were assessed in a minimum of 6 studies using the same methodology. In this case, a separate bivariate model was estimated for each of the cut-off points, and each study could contribute to one or more of the models depending on what cut-off points it reported [21 ].
The following variables were assessed as possible sources of heterogeneity: a) the study setting; b) the measure used as the gold standard; c) the version of the instrument (English versus cultural adaptation); d) the age group of the study sample; e) disorder prevalence; and f) specific QUADAS items for which more than 20% of the studies presented problems. Heterogeneity was evaluated with the Rutter and Gatsonis mixed effects models (see above) including each covariate at a time and testing its statistical significance with the likelihood ratio test. Estimates of model parameters were obtained using the METADAS macro [29 ] implemented in SAS (SAS v9.1.2) [30 ].
The suggested cut-point for the scale is 16, but many studies presented accuracy results of the CES-D using other cut-points, since they aimed to evaluate and compare their performance and select the optimal value in that specific population. However, for the assessment of overall performance of the scale using information from all the 28 studies, we selected results for only one cut-off point per study, so that each study contributes only one estimate of sensitivity and specificity as required by the statistical methods applied[21 ]. We chose the cut-off point of 16 whenever possible, as this is the value usually recommended for the detection of depression with the CES-D. Notwithstanding, when a study did not report diagnostic accuracy results for cut-off 16, we used the cut-off point reported in that particular study. When more than one cut-off point was reported in an article, and in order to avoid multiple testing effects, we selected the cut-off with the best diagnostic accuracy within the study. We obtained the coupled forest plot reporting the raw data consisting of the 2x2 sensitivity and specificity table from each study, as well as the estimated sensitivity (SN; the proportion of true cases correctly classified by the cut-off point) and specificity (SP; the proportion of true non-cases correctly classified) for detection of depression of each of the studies, together with 95% confidence intervals. In the context of meta-analysis, when a variety of sensitivity and specificity values for a given test are available from several independent studies depending on the cut-off point, the summary receiver operating characteristic (SROC) has been proposed as a way to assess diagnostic data [25 (link)]. The SROC curve considers both sensitivity and specificity and the relationship between them, taking into account that not all studies used the same cut-off. It is assumed that different values of sensitivity and specificity apply if the cut-off point defining a positive test result varies from study to study, everything else being equal. Several procedures have been proposed to estimate the SROC curve from a set of independent studies [25 (link)–27 (link)]. Here, the Rutter and Gatsonis mixed effects model [26 (link)] was fitted to estimate the SROC curve, and the sensitivity and specificity of each study, weighted by study size were plotted in the ROC space. The area under the curve (AUC) for the fitted SROC was computed from the estimated diagnostic odds ratio (DOR) following the method described by Walter [28 (link)]. Also, for the subsample of studies that provided diagnostic accuracy results for the cut-off point 16 (n = 22), we estimated a bivariate meta-regression [27 (link)], which allowed us to obtain pooled estimates of a range of summary performance measures of the test’s ability to detect the presence of a disease for a given cut-off point. Specifically, the summary measures obtained were: a) specificity and sensitivity, and their corresponding 95% confidence intervals; b) the positive likelihood ratio (LR+) that described how many times more likely positive test results were in the diseased group compared to the non-diseased group; c) the negative likelihood ratio (LR-), describing how many times less likely negative test results were in the diseased group compared to the non-diseased group; and d) the DOR, that summarizes the diagnostic accuracy of the test as a single number describing how many times higher the odds are of obtaining a positive test result in a diseased rather than in a non-diseased person. Additionally, we evaluated the screening accuracy of other cut-off points that were assessed in a minimum of 6 studies using the same methodology. In this case, a separate bivariate model was estimated for each of the cut-off points, and each study could contribute to one or more of the models depending on what cut-off points it reported [21 ].
The following variables were assessed as possible sources of heterogeneity: a) the study setting; b) the measure used as the gold standard; c) the version of the instrument (English versus cultural adaptation); d) the age group of the study sample; e) disorder prevalence; and f) specific QUADAS items for which more than 20% of the studies presented problems. Heterogeneity was evaluated with the Rutter and Gatsonis mixed effects models (see above) including each covariate at a time and testing its statistical significance with the likelihood ratio test. Estimates of model parameters were obtained using the METADAS macro [29 ] implemented in SAS (SAS v9.1.2) [30 ].
Free full text: Click here
