Improving LLM-Based Health Information Extraction with In-Context Learning

The Large Language Model (LLM) has received widespread attention in the industry. In the context of the popularity of LLM, almost all NLP tasks are transformed into prompt based language generation tasks. On the other hand, LLM can also achieve superior results on brand new tasks without fine-tuning, solely with a few in-context examples. This paper describes our participation in the China Health Information Processing Conference (CHIP 2023). We focused on in-context learning (ICL) and experimented with different combinations of demonstration retrieval strategies on the given task and tested the optimal strategy combination proposed by us. The experimental results show that our retrieval strategies based on Chinese-LlaMA2-13B-chat achieved a average score of 40.27, ranked the first place among five teams, confirmed the effectiveness of our method.