A closed max‐t test for multiple comparisons of areas under the ROC curve

Abstract Comparing areas under the ROC curve (AUCs) is a popular approach to compare prognostic biomarkers. The aim of this paper is to present an efficient method to control the family‐wise error rate when multiple comparisons are performed. We suggest to combine the max‐t test and the closed testing procedures. We build on previous work on asymptotic results for ROC curves and on general multiple testing methods to efficiently take into account both the correlations between the test statistics and the logical constraints between the null hypotheses. The proposed method results in an uniformly more powerful procedure than both the single‐step max‐t test procedure and popular stepwise extensions of the Bonferroni procedure, such as Bonferroni–Holm. As demonstrated in this paper, the method can be applied in most usual contexts, including the time‐dependent context with right censored data. We show how the method works in practice through a motivating example where we compare several psychometric scores to predict the t‐year risk of Alzheimer's disease. The example illustrates several multiple testing settings and demonstrates the advantage of using the proposed methods over common alternatives. R code has been made available to facilitate the use of the methods by others.