“一刀切”可行吗？：多文档摘要领域迁移中的失败问题衡量
Can one size fit all?: Measuring Failure in Multi-Document Summarization Domain Transfer
AI总结：
该研究针对多文档摘要的零样本领域迁移失败问题，跨训练方法、领域和多维度评估四类MDS模型，定义了迁移失败的判定标准，同时探究了直接套用现有摘要指标的潜在问题。
摘要：
生成式多文档摘要（MDS）是自动汇总多篇文档中信息的任务，文档范围涵盖新闻文章到多说话人对话。当前MDS模型的训练方法可分为四类：带特殊预训练的端到端方法（“直接法”）、分块后摘要、抽取后摘要，以及使用GPT风格模型进行推理。在本研究中，我们跨训练方法、领域和维度（参考相似度、质量、事实性）评估MDS模型，以分析在零样本领域迁移设置下，在某一领域（新闻、科学、对话）训练的模型为何以及如何无法对另一领域的文档生成摘要。我们将领域迁移“失败”定义为事实性下降、与目标的偏差增大，以及摘要质量整体下降。除了探索MDS模型的领域迁移问题，我们还研究了直接套用流行摘要指标可能存在的问题。
英文摘要：
Abstractive multi-document summarization (MDS) is the task of automatically summarizing information in multiple documents, from news articles to conversations with multiple speakers. The training approaches for current MDS models can be grouped into four approaches: end-to-end with special pre-training ("direct"), chunk-then-summarize, extract-then-summarize, and inference with GPT-style models. In this work, we evaluate MDS models across training approaches, domains, and dimensions (reference similarity, quality, and factuality), to analyze how and why models trained on one domain can fail to summarize documents from another (News, Science, and Conversation) in the zero-shot domain transfer setting. We define domain-transfer "failure" as a decrease in factuality, higher deviation from the target, and a general decrease in summary quality. In addition to exploring domain transfer for MDS models, we examine potential issues with applying popular summarization metrics out-of-the-box.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
