A Simple Divide-and-Conquer-based Distributed Method for the Accelerated Failure Time Model

AbstractThe accelerated failure time (AFT) model is an appealing tool in survival analysis because of its ease of interpretation, but when there is a large volume of data, fitting an AFT model and carrying out the associated inference on one computer can be computationally demanding. This poses a severe limitation for the application of the AFT model in the face of big data. The article addresses this problem by developing a simple distributed method for estimating the parameters of an AFT model based on the divide-and-conquer strategy, which has the dual benefits of statistical efficiency and computational economy. It is an iterative method that involves, for the most part, some rather simple algebraic operations, except for obtaining the initial estimate, which is based on a smoothed approximation of the Gehan estimating equation. Our results show that the proposed method yields estimates that converge after a few iterations and an estimator that is asymptotically as efficient as the benchmark estimator obtained by using the full data in one go. We also develop an associated inference procedure. The merits of the proposed method are demonstrated via an extensive simulation study. The method is applied to a kidney transplantation dataset. Supplementary materials for this article are available online.KEYWORDS: Accelerated failure time modelAlgorithmBig dataDistributed inferenceDivide-and-conquerGehan estimating equation Supplementary MaterialsR code: We provide R code to replicate the simulation studies.Appendix: Theoretical proofs of Theorems 1–4 are provided in the appendix.AcknowledgementWe thank the Editor, Associate Editor and referees for comments and suggestions on an earlier version of this paper. All remaining errors are ours.Disclosure StatementNo potential conflict of interest was reported by the author.Additional informationFundingWan's work was supported by the Hong Kong Research Grant Council (CityU-11501522) and the National Natural Science Foundation of China (72273120). Zhou's work is supported by the National Key Research and Development Program of China (2021YFA1000100 and 2021YFA1000101) and the State Key Program of National Natural Science Foundation of China (71931004).