Identifying Exceptional Drug Responses

To objectively identify compounds whose response distributions show exceptional positive response, that is, a relatively few highly responsive samples at the right tail of the drug response distribution, we calculated the sample skewness γ of the drugs' empirical response distribution over all the samples under analysis. The one-sided significance p-value of the observed positive skewness was assessed using the D'Agostino15 test in the R-package “moments” (version 0.13, http://cran.r-project.org/package=moments). This enables systematic detection of drug-sensitive patient sub-groups for a given compound, without visually going through all the drug response distributions. When comparing two sets of samples, such as highly responsive patient samples against the remaining samples for those compounds initially identified with positive skewness, we assessed the difference in the response levels between the two pre-defined sample groups with the Wilcoxon rank-sum test. We chose to use the non-parametric test because the response distributions cannot be assumed to be normally distributed.
The predictive accuracy of the DSS, IC₅₀ and AA metrics was assessed in terms of their capability to distinguish the active dose-response curves from the inactive ones using the receiver operating characteristic (ROC) analyses; ROC curves evaluate the relative trade-off between true positive rate (sensitivity) and false positive rate (1 – specificity) of the metric when ordering the dose-response curves according to the increasing value of the response metric16 (link). The overall accuracy of each response metric was summarized using the area under the ROC curve (AUROC) measure; for an ideal metric, AUROC = 1, whereas a random metric obtains an AUROC = 0.5 on average. Statistical significance of an observed AUROC, when compared to random classifier, was assessed using the roc.area function in the R-package “verification”. Statistical significance of an observed AUROC difference between two response metrics was assessed using the “pROC” package with the De Long's test17 (link).