Deep representation learning for clustering longitudinal survival data from electronic health records

Abstract Precision medicine can be defined as providing the right treatment to the right patient at the right time, and it requires the ability to identify clinically relevant patient subgroups with high accuracy. The increasing availability of large-scale longitudinal electronic health records (EHR) datasets has provided major opportunities for artificial intelligence and machine learning in mining such complex datasets for identifying novel disease subtypes. However, disease subtypes often exist in the context of certain disease-relevant risk events, and current efforts have been limited by stratifying patients either only by disease trajectory or only by event risk, largely ignoring the interactions between the two, which can result in subgroups that still display great heterogeneity in event risk and/or underlying molecular mechanisms. To capture such interactions, novel methods are needed that allow for clustering patients simultaneously by disease trajectory and event risk, To address this current gap in the literature, we developed TransVarSur (Transformer Variational Survival modeling). TransVarSur integrates a Transformer-based Gaussian mixture variational autoencoder with time-to-event modeling to capture complex relationships between cluster-specific EHR trajectories and survival times. We validated TransVarSur by showing superior performance relative to a range of baseline methods that either ignore longitudinality or the interactions between disease trajectories and event risk, on both synthetic and real-world benchmark datasets with known ground-truth clustering. We then applied TransVarSur to 1908 Crohn's disease patients from the UK Biobank and successfully identified four clusters displaying both divergent EHR trajectories and divergent progression towards the risk event intestinal obstruction. A further analysis of the clusters revealed known clinical and genetic factors relevant in Crohn's disease and progression to intestinal obstruction. In conclusion, we demonstrated TransVarSur’s ability to disentangle interactions between disease trajectories and risk events to more accurately stratify a patient population into clinically and genetically relevant subgroups. Hence, it can be a powerful tool in the development of precision medicine approaches.