AI-Based multimodal Multi-tasks analysis reveals tumor molecular heterogeneity, predicts preoperative lymph node metastasis and prognosis in papillary thyroid carcinoma: A retrospective study

Background: Papillary thyroid carcinoma (PTC) is the predominant form of thyroid cancer globally, especially when lymph node metastasis (LNM) occurs. Molecular heterogeneity, driven by genetic alterations and tumor microenvironment components, contributes to the complexity of PTC. Understanding these complexities is essential for precise risk stratification and therapeutic decisions. Methods: This study involved a comprehensive analysis of 521 patients with PTC from our hospital and 499 patients from The Cancer Genome Atlas (TCGA). The real-world cohort 1 comprised 256 patients with stage I–III PTC. Tissues from 252 patients were analyzed by DNA-based next-generation sequencing, and tissues from four patients were analyzed by single-cell RNA sequencing (scRNA-seq). Additionally, 586 PTC pathological sections were collected from TCGA, and 275 PTC pathological sections were collected from the real-world cohort 2. A deep learning multimodal model was developed using matched histopathology images, genomic, transcriptomic, and immune cell data to predict LNM and disease-free survival (DFS). Results: This study included a total of 1,011 PTC patients, comprising 256 patients from cohort 1, 275 patients from cohort 2, and 499 patients from TCGA. In cohort 1, we categorized PTC into four molecular subtypes based on BRAF, RAS, RET, and other mutations. BRAF mutations were significantly associated with LNM and impacted DFS. ScRNA-seq identified distinct T cell subtypes and reduced B cell diversity in BRAF-mutated PTC with LNM. The study also explored cancer-associated fibroblasts and macrophages, highlighting their associations with LNM. The deep learning model was trained using 405 pathology slides and RNA sequences from 328 PTC patients and validated with 181 slides and RNA sequences from 140 PTC patients in the TCGA cohort. It achieved high accuracy, with an AUC of 0.86 in the training cohort, 0.84 in the validation cohort, and 0.83 in the real-world cohort 2. High-risk patients in the training cohort had significantly lower DFS rates ( P <0.001). Model AUCs were 0.91 at 1 year, 0.93 at 3 years, and 0.87 at 5 years. In the validation cohort, high-risk patients also had lower DFS ( P <0.001); the AUCs were 0.89, 0.87, and 0.80 at 1, 3, and 5 years. We utilized the GradCAM algorithm to generate heatmaps from pathology-based deep learning models, which visually highlighted high-risk tumor areas in PTC patients. This enhanced clinicians’ understanding of the model’s predictions and improved diagnostic accuracy, especially in cases with lymph node metastasis. Conclusion: The AI-based analysis uncovered vital insights into PTC molecular heterogeneity, emphasizing BRAF mutations’ impact. The integrated deep learning model shows promise in predicting metastasis, offering valuable contributions to improved diagnostic and therapeutic strategies.