arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2407.10486cs.AIcs.CL

IDEAL: 利用大型语言模型的无限和动态特性进行查询导向的摘要

IDEAL: Leveraging Infinite and Dynamic Characterizations of Large Language Models for Query-focused Summarization

Zhejiang University(浙江大学);Tencent, Wechat(腾讯,微信)

Jie Cao, Dian Jiao, Yang Dai, Rolan Yan, Wenqiao Zhang, Siliang Tang

AI总结:

本文针对查询导向摘要问题,提出两种核心方法:高效细粒度查询-LLM对齐和长文档摘要,通过Query-aware HyperExpert和Query-focused Infini-attention模块实现,实验验证了方法的有效性和通用性。

摘要:

查询导向摘要(QFS)旨在生成回答特定问题的摘要,使用户能够更好地控制和个性化内容。随着大型语言模型(LLMs)的出现,其通过大规模预训练展现出了强大的文本理解能力,这表明了提取片段生成的巨大潜力。本文系统地研究了LLMs基于QFS模型应具备的两个不可或缺特性,即高效细粒度查询-LLM对齐和长文档摘要。相应地,我们提出了两个模块,称为Query-aware HyperExpert和Query-focused Infini-attention,以访问上述特性。这些创新为QFS技术的更广泛应用和可访问性铺平了道路。在现有QFS基准上的广泛实验表明了所提出方法的有效性和通用性。

英文摘要:

Query-focused summarization (QFS) aims to produce summaries that answer particular questions of interest, enabling greater user control and personalization. The advent of large language models (LLMs), shows their impressive capability of textual understanding through large-scale pretraining, which implies the great potential of extractive snippet generation. In this paper, we systematically investigated two indispensable characteristics that the LLMs-based QFS models should be harnessed, \emph{Efficiently Fine-grained Query-LLM Alignment} and \emph{Lengthy Document Summarization}, respectively. Correspondingly, we propose two modules called Query-aware HyperExpert and Query-focused Infini-attention to access the aforementioned characteristics. These innovations pave the way for broader application and accessibility in the field of QFS technology. Extensive experiments conducted on existing QFS benchmarks indicate the effectiveness and generalizability of the proposed approach.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑