Application Scheme of Clinical Trial Questionnaire Pre Recruitment Integrating LLM and Knowledge Graph

Background: The establishment of inclusion and exclusion criteria, which serve to identify participants aligning with specific research requirements, plays a pivotal role in safeguarding both the safety and research validity of clinical trials. Currently, the recruitment process for clinical trials is relatively passive, resulting in a failure to promptly attain a sufficient number of eligible participants. In recent years, the development of large language models (LLMs) and knowledge graphs has presented new approaches to pre-screening and recruitment for clinical trials, facilitating the optimization of recruitment efficiency and increasing participant engagement.Method: This paper proposes an application scheme for the pre-recruitment phase of clinical trials, leveraging the technical advantages of knowledge graphs and large language models (LLMs). The introduction of LLM into the pre-recruitment stage significantly enhances the system's intelligence. The application scheme encompasses the automated generation of pre-recruitment questionnaires, automatic assessment of candidate eligibility based on inclusion and exclusion criteria, and the provision of knowledge-based question and answer services related to clinical medical terminology.Results: ChatGLM-130B and ChatGPT-3.5 have demonstrated exceptional proficiency in the generation of questionnaires. 6.89% of questionnaires generated by ChatGLM-130B manifested issues related to JSON output formatting, while 3.44% of questionnaires generated by ChatGPT-3.5 exhibited duplicate questions. The accuracy rates for evaluating questionnaire responses were 90.47% for ChatGLM-130B and 91.66% for ChatGPT-3.5. The application has been implemented in the pilot phase at Shanghai University of Traditional Chinese Medicine, Longhua Hospital.Summary: This solution has automated the process from questionnaire generation to patient eligibility determination, which can significantly enhance recruitment efficiency.