The utility of Vision Transformer in preoperatively predicting microvascular invasion status of hepatocellular carcinoma

Microvascular invasion (MVI) is a risk factor for early recurrence and poor prognosis of hepatocellular carcinoma (HCC). Preoperative assessment of MVI status is beneficial for clinical therapy and prognosis evaluation.A total of 305 surgically resected patients were included retrospectively. All recruited patients underwent plain and contrast-enhanced abdominal CT. They were then randomly divided into training and validation sets in a ratio of 8:2. Self-attention-based ViT-B/16 and ResNet-50 analyzed CT images to predict MVI status preoperatively. Then, Grad-CAM was used to generate an attention map showing the high-risk MVI patches. Using five-fold cross validation, the performance of each model was evaluated.Among 305 HCC patients, 99 patients were pathologically MVI-positive and 206 were MVI-negative. ViT-B/16 with fusion phase predicted the MVI status with an AUC of 0.882 and an accuracy of 86.8% in the validation set, which is similar to ResNet-50 with an AUC of 0.875 and an accuracy of 87.2%. The fusion phase improved performance a bit as compared to the single phase used for MVI prediction. The influence of peritumoral tissue on predictive ability was limited. A color visualization of the suspicious patches where microvascular has invaded was presented by attention maps.ViT-B/16 model can predict preoperative MVI status in CT images of HCC patients. Assisted by attention maps, it can assist patients in making tailored treatment decisions.