VetLLM: Large Language Model for Predicting Diagnosis from Veterinary Notes

Biocomputing 2024, pp. 120-133 (2023) Open AccessVetLLM: Large Language Model for Predicting Diagnosis from Veterinary NotesYixing Jiang, Jeremy A. Irvin, Andrew Y. Ng, and James ZouYixing JiangStanford University, Stanford, CA, United States, Jeremy A. IrvinStanford University, Stanford, CA, United States, Andrew Y. NgStanford University, Stanford, CA, United States, and James ZouStanford University, Stanford, CA, United Stateshttps://doi.org/10.1142/9789811286421_0010Cited by:0 (Source: Crossref) PreviousNext AboutSectionsPDF/EPUB ToolsAdd to favoritesDownload CitationsTrack CitationsRecommend to Library ShareShare onFacebookTwitterLinked InRedditEmail Abstract: Lack of diagnosis coding is a barrier to leveraging veterinary notes for medical and public health research. Previous work is limited to develop specialized rule-based or customized supervised learning models to predict diagnosis coding, which is tedious and not easily transferable. In this work, we show that open-source large language models (LLMs) pretrained on general corpus can achieve reasonable performance in a zero-shot setting. Alpaca-7B can achieve a zero-shot F1 of 0.538 on CSU test data and 0.389 on PP test data, two standard benchmarks for coding from veterinary notes. Furthermore, with appropriate fine-tuning, the performance of LLMs can be substantially boosted, exceeding those of strong state-of-the-art supervised models. VetLLM, which is fine-tuned on Alpaca-7B using just 5000 veterinary notes, can achieve a F1 of 0.747 on CSU test data and 0.637 on PP test data. It is of note that our fine-tuning is data-efficient: using 200 notes can outperform supervised models trained with more than 100,000 notes. The findings demonstrate the great potential of leveraging LLMs for language processing tasks in medicine, and we advocate this new paradigm for processing clinical text. Keywords: Diagnosis ExtractionVeterinary NotesVeterinary MedicineLarge Language ModelsLLMFoundation Models FiguresReferencesRelatedDetails Recommended Biocomputing 2024Metrics History Information© The AuthorsOpen Access chapter published by World Scientific Publishing Company and distributed under the terms of the Creative Commons Attribution Non-Commercial (CC BY-NC) 4.0 License.KeywordsDiagnosis ExtractionVeterinary NotesVeterinary MedicineLarge Language ModelsLLMFoundation ModelsPDF download