自动汇总
计算机科学
情报检索
自然语言处理
人工智能
万维网
作者
Pavlos Zakkas,Suzan Verberne,Jakub Zavrel
标识
DOI:10.1007/978-3-031-56027-9_23
摘要
We propose a prompt-based pipeline for extreme summarization of large collections of scientific articles, which facilitates the consumption of scientific knowledge in high-volume fast-paced fields like AI. Although prompting of generative large language models (LLMs) has been applied to news summarization, its effectiveness in the scientific domain and in multi-document summarization is underexplored. We propose a three-step approach for summarizing a large collection of documents (e.g. hundreds or thousands of papers published in a conference). First, selecting representative papers per document cluster, second, performing single-document summarization (SDS) of the selected papers, and third, aggregating these in a multi-document summarization (MDS) step. Both the single-document summaries and the multi-document summaries are generated with an instruction-tuned LLM. The cluster summaries are used to generate a blog post summarizing a conference. We show that our SDS model achieves better results than strong fine-tuned models on the SciTLDR benchmark. Our two-step approach reaches the performance of state-of-the-art fine-tuned MDS models on the Multi-XScience benchmark. Through a small-scale user study, we find that , although a human-written blog post is clearly preferred over an automatically generated one, the users appreciate the good informativeness and factuality of our pipeline. Our findings demonstrate the potential use of generative LLMs as a way to digest large amounts of scientific papers and help researchers to stay up-to-date with rapidly evolving fields.
科研通智能强力驱动
Strongly Powered by AbleSci AI