Collaborative multi-modal deep learning and radiomic features for classification of strokes within 6 h

Clinicians use imaging-based acute stroke onset time (SOT) to make crucial decisions regarding stroke treatments, such as thrombolysis or thrombectomy. Patients may receive intravenous thrombolysis and undergo endovascular thrombectomy (EVT) within 3 or 4.5 h and 6 h from SOT, respectively. Most of the classification algorithms developed so far classify SOT within 4.5 h for thrombolysis. In this study, we demonstrated a deep learning (DL) method to classify SOT within 6 h to identify patients requiring EVT. We developed a DL-based segmentation model using a multi-modal UNet (MM-UNet) to predict the region of interest (ROI) from magnetic resonance (MR) images. Radiomic features were extracted from the MR images and ROI. Additionally, we proposed a DL model to extract hidden representations (deep features) using the MM-UNet and ResNet-18 models. We found that the classification performance improved by combining radiomic and deep features. The cross-validation results indicate that our proposed method sufficiently classified SOT within 6 h, achieving an F0.5 score of 80.6%. The DL model using multi-modal MR images can potentially become a practical decision-support tool for stroke treatments.