arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.05360cs.CL
收藏

通过专业知识分离实现灵活且可适应的摘要生成

Flexible and Adaptable Summarization via Expertise Separation

KAUST(阿卜杜拉国王科技大学);MBZUAI(Mohamed bin Zayed 人工智能大学);Ant Group(蚂蚁集团);Shandong University(山东大学);Peking University(北京大学);Renmin University of China(中国人民大学);University of Notre Dame(圣母大学)

Xiuying Chen, Mingzhe Li, Shen Gao, Xin Cheng, Qingqing Zhu, Rui Yan, Xin Gao, Xiangliang Zhang

AI总结:

本文提出专家混合摘要架构MoeSumm,通过主专家与副专家的分工及最大间隔损失,在参数高效前提下同时实现多领域摘要灵活性与域外少样本/零样本适应性,并在11个数据集上验证其优于基线和大语言模型。

摘要:

一个熟练的摘要模型应同时具备灵活性——处理一系列域内摘要任务的能力,以及适应性——获取新知识并适应未见过的域外任务的能力。与通过参数扩展实现这一目标的大语言模型(LLMs)不同,本研究提出了一种更具参数效率的方法。我们的动机基于以下原则:捕获显著信息的一般摘要能力可以在不同任务之间共享,而特定领域的摘要能力则需要相互区分并量身定制。具体而言,我们提出了MoeSumm,一种专家混合摘要架构,它利用一个主专家来获得一般摘要能力,并利用副专家有选择地协作以满足特定摘要任务的要求。我们进一步提出了一种最大间隔损失来促进这些能力的分离。我们的模型对一般摘要能力和特定领域摘要能力的明确分离,使其在保持参数效率的同时,具有显著的灵活性和适应性。MoeSumm通过使用共享的主专家和选定的副专家,以单一模型管理多个领域的摘要任务,从而实现灵活性。它通过定制副专家来适应域外少样本和零样本场景,从而表现出适应性。在11个数据集上的实验结果表明,与近期基线和大语言模型相比,我们的模型具有优越性。我们还提供了统计和可视化证据,证明MoeSumm中两种能力的明确分离(https://github.com/iriscxy/MoE_Summ)。

英文摘要:

A proficient summarization model should exhibit both flexibility -- the capacity to handle a range of in-domain summarization tasks, and adaptability -- the competence to acquire new knowledge and adjust to unseen out-of-domain tasks. Unlike large language models (LLMs) that achieve this through parameter scaling, we propose a more parameter-efficient approach in this study. Our motivation rests on the principle that the general summarization ability to capture salient information can be shared across different tasks, while the domain-specific summarization abilities need to be distinct and tailored. Concretely, we propose MoeSumm, a Mixture-of-Expert Summarization architecture, which utilizes a main expert for gaining the general summarization capability and deputy experts that selectively collaborate to meet specific summarization task requirements. We further propose a max-margin loss to stimulate the separation of these abilities. Our model's distinct separation of general and domain-specific summarization abilities grants it with notable flexibility and adaptability, all while maintaining parameter efficiency. MoeSumm achieves flexibility by managing summarization across multiple domains with a single model, utilizing a shared main expert and selected deputy experts. It exhibits adaptability by tailoring deputy experts to cater to out-of-domain few-shot and zero-shot scenarios. Experimental results on 11 datasets show the superiority of our model compared with recent baselines and LLMs. We also provide statistical and visual evidence of the distinct separation of the two abilities in MoeSumm (https://github.com/iriscxy/MoE_Summ).

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑