Prediction of soft tissue sarcoma response to radiotherapy using longitudinal diffusion MRI and a deep neural network with generative adversarial network‐based data augmentation

Purpose The goal of this study was to predict soft tissue sarcoma response to radiotherapy (RT) using longitudinal diffusion‐weighted MRI (DWI). A novel deep‐learning prediction framework along with generative adversarial network (GAN)‐based data augmentation was investigated for the response prediction. Methods Thirty soft tissue sarcoma patients who were treated with five‐fraction hypofractionated radiation therapy (RT, 6Gy×5) underwent diffusion‐weighted MRI three times throughout the RT course using an MR‐guided radiotherapy system. Pathologic treatment effect (TE) scores, ranging from 0‐100%, were obtained from the post‐RT surgical specimen as a surrogate of patient treatment response. Patients were divided into three classes based on the TE score (TE ≤ 20%, 20% < TE < 90%, TE ≥ 90%). Apparent diffusion coefficient (ADC) maps of the tumor from the three time points were combined as 3‐channel images. An auxiliary classifier generative adversarial network (ACGAN) was trained on 20 patients to augment the data size. A total of 15,000 synthetic images were generated for each class. A prediction model based on a previously described VGG‐19 network was trained using the synthesized data, validated on five unseen validation patients, and tested on the remaining five test patients. The entire process was repeated seven times, each time shuffling the training, validation, and testing datasets such that each patient was tested at least once during the independent test stage. Prediction performance for slice‐based prediction and patient‐based prediction was evaluated. Results The average training and validation accuracies were 86.5% ± 1.6% and 84.8% ± 1.8%, respectively, indicating that the generated samples were good representations of the original patient data. Among the seven rounds of testing, slice by slice prediction accuracy ranged from 81.6% to 86.8%. The overall accuracy of the independent test sets was 83.3%. For patient‐based prediction, 80% was achieved in one round and 100% was achieved in the remaining six rounds. The mean accuracy was 97.1%. Conclusion This study demonstrated the potential to use deep learning to predict the pathologic treatment effect from longitudinal DWI. Accuracies of 83.3% and 97.1% were achieved on independent test sets for slice‐based and patient‐based prediction respectively.