Symptom-BERT: Enhancing Cancer Symptom Detection in EHR Clinical Notes

Context Extracting cancer symptom documentation allows clinicians to develop highly individualized symptom prediction algorithms to deliver symptom management care. Leveraging advanced language models to detect symptom data in clinical narratives can significantly enhance this process. Objective This study uses a pre-trained large language model to detect and extract cancer symptoms in clinical notes. Methods We developed a pre-trained language model to identify cancer symptoms in clinical notes based on a clinical corpus from the Enterprise Data Warehouse for Research at a healthcare system in the Midwestern United States. This study was conducted in 4 phases: 1 Sung H Ferlay J Siegel RL et al. Global Cancer Statistics 2020: GLOBOCAN Estimates of Incidence and Mortality Worldwide for 36 Cancers in 185 Countries. CA Cancer J Clin. 2021; 71: 209-249 Crossref PubMed Scopus (55646) Google Scholar pre-training a Bio-Clinical BERT model on 1 million unlabeled clinical documents, 2 Siegel RL Miller KD Wagle NS Jemal A. Cancer statistics, 2023. CA Cancer J Clin. 2023; 73: 17-48 Crossref PubMed Scopus (5090) Google Scholar fine-tuning Symptom-BERT for detecting 13 cancer symptom groups within 1112 annotated clinical notes, 3 Lizán L Pérez-Carbonell L Comellas M. Additional Value of Patient-Reported Symptom Monitoring in Cancer Care: A Systematic Review of the Literature. Cancers (Basel). 2021; 13 Google Scholar generating 180 synthetic clinical notes using ChatGPT-4 for external validation, and 4 Tripp-Reimer T Williams JK Gardner SE et al. An integrated model of multimorbidity and symptom science. Nurs Outlook. 2020; 68: 430-439 Abstract Full Text Full Text PDF PubMed Scopus (15) Google Scholar comparing the internal and external performance of Symptom-BERT against a non-pre-trained version and six other BERT implementations. Results The Symptom-BERT model effectively detected cancer symptoms in clinical notes. It achieved results with a micro-averaged F1-score of 0.933, an AUC of 0.929 internally, and 0.831 and 0.834 externally. Our analysis shows that physical symptoms, like Pruritus, are typically identified with higher performance than psychological symptoms, such as Anxiety. Conclusion This study underscores the transformative potential of specialized pre-training on domain-specific data in boosting the performance of language models for medical applications. The Symptom-BERT model's exceptional efficacy in detecting cancer symptoms heralds a groundbreaking stride in patient-centered AI technologies, offering a promising path to elevate symptom management and cultivate superior patient self-care outcomes.