Real time volumetric MRI for 3D motion tracking via geometry‐informed deep learning

Abstract Purpose To develop a geometry‐informed deep learning framework for volumetric MRI with sub‐second acquisition time in support of 3D motion tracking, which is highly desirable for improved radiotherapy precision but hindered by the long image acquisition time. Methods A 2D–3D deep learning network with an explicitly defined geometry module that embeds geometric priors of the k‐space encoding pattern was investigated, where a 2D generation network first augmented the sparsely sampled image dataset by generating new 2D representations of the underlying 3D subject. A geometry module then unfolded the 2D representations to the volumetric space. Finally, a 3D refinement network took the unfolded 3D data and outputted high‐resolution volumetric images. Patient‐specific models were trained for seven abdominal patients to reconstruct volumetric MRI from both orthogonal cine slices and sparse radial samples. To evaluate the robustness of the proposed method to longitudinal patient anatomy and position changes, we tested the trained model on separate datasets acquired more than one month later and evaluated 3D target motion tracking accuracy using the model‐reconstructed images by deforming a reference MRI with gross tumor volume (GTV) contours to a 5‐min time series of both ground truth and model‐reconstructed volumetric images with a temporal resolution of 340 ms. Results Across the seven patients evaluated, the median distances between model‐predicted and ground truth GTV centroids in the superior‐inferior direction were 0.4 ± 0.3 mm and 0.5 ± 0.4 mm for cine and radial acquisitions, respectively. The 95‐percentile Hausdorff distances between model‐predicted and ground truth GTV contours were 4.7 ± 1.1 mm and 3.2 ± 1.5 mm for cine and radial acquisitions, which are of the same scale as cross‐plane image resolution. Conclusion Incorporating geometric priors into deep learning model enables volumetric imaging with high spatial and temporal resolution, which is particularly valuable for 3D motion tracking and has the potential of greatly improving MRI‐guided radiotherapy precision.