面向文档摘要的关键要素引导的小规模大语言模型调优
Key-Element-Informed sLLM Tuning for Document Summarization
AI总结：
针对小规模大语言模型在长文档摘要中关键信息缺失、相关性低的问题，提出关键要素引导的指令调优方法KEITSum，通过识别文档关键要素指导摘要生成，在对话和新闻数据集上取得与专有LLM相当的高质量摘要效果。
摘要：
大语言模型（LLMs）的显著进展已使高质量文本摘要成为可能。然而，这种能力目前仅能通过规模庞大的LLMs或需付费的专有LLMs获得。为此，易于获取且成本低廉的小规模大语言模型（sLLMs）已被广泛研究，但它们常常遗漏关键信息和实体，即相关性较低，尤其是在输入文档较长时。因此，我们提出了一种面向摘要的关键要素引导的指令调优方法，称为KEITSum，该方法识别文档中的关键要素，并指导sLLM生成涵盖这些关键要素的摘要。在对话和新闻数据集上的实验结果表明，采用KEITSum的sLLM确实能提供高质量摘要，具有更高的相关性和更少的幻觉，可与专有LLM相媲美。
英文摘要：
Remarkable advances in large language models (LLMs) have enabled high-quality text summarization. However, this capability is currently accessible only through LLMs of substantial size or proprietary LLMs with usage fees. In response, smaller-scale LLMs (sLLMs) of easy accessibility and low costs have been extensively studied, yet they often suffer from missing key information and entities, i.e., low relevance, in particular, when input documents are long. We hence propose a key-element-informed instruction tuning for summarization, so-called KEITSum, which identifies key elements in documents and instructs sLLM to generate summaries capturing these key elements. Experimental results on dialogue and news datasets demonstrate that sLLM with KEITSum indeed provides high-quality summarization with higher relevance and less hallucinations, competitive to proprietary LLM.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
