Deep Learning Approaches for End-to-End Modeling of Medical Spatiotemporal Data

For many medical applications, a single, stationary image may not be sufficient for detecting subtle pathology. Advancements in fields such as computer vision have produced robust deep learning (DL) techniques able to effectively learn complex interactions between space and time for prediction. This chapter presents an overview of different medical applications of spatiotemporal DL for prognostic and diagnostic predictive tasks, and how they built on important advancements in DL from other domains. Although many of the current approaches draw heavily from previous works in other fields, adaptation to the medical domain brings unique challenges, which will be discussed, along with techniques being used to address them. Although the use of spatiotemporal DL in medical applications is still relatively new, and lags behind the progress seen from still images, it provides unique opportunities to incorporate information about functional dynamics into prediction, which could be vital in many medical applications. Current medical applications of spatiotemporal DL have demonstrated the potential of these models, and recent advancements make this space poised to produce state-of-the-art models for many medical applications.