UniSumm 与 SummZoo:面向少样本摘要的统一模型与多样化基准
UniSumm and SummZoo: Unified Model and Diverse Benchmark for Few-Shot Summarization
AI总结:
本文提出统一少样本摘要模型 UniSumm,通过多任务预训练与前缀微调提升跨任务泛化,并发布多样化基准 SummZoo,实验证明其显著优于强基线且接近 GPT-3.5。
摘要:
高昂的标注成本和各类摘要任务的多样化需求推动了少样本摘要的发展。然而,尽管许多摘要任务和数据集不断涌现,当前少样本摘要系统的训练范式忽略了异构数据集中潜在可共享的知识。为此,我们提出了 UniSumm,一个统一的少样本摘要模型,该模型在多个摘要任务上进行预训练,并可通过前缀微调在任何少样本摘要任务上取得优异表现。同时,为了在多样性和鲁棒性原则下更好地评估少样本摘要器,我们组装并发布了一个新的基准 SummZoo。它包含 8 个摘要任务,每个任务配有多组少样本样例,覆盖不同领域。实验结果与分析表明,在 SummZoo 的所有子任务上,无论自动评估还是人工评估,UniSumm 都以较大幅度超越强基线,并且在人工评估中取得了与 GPT-3.5 模型相当的结果。
英文摘要:
The high annotation costs and diverse demands of various summarization tasks motivate the development of few-shot summarization. However, despite the emergence of many summarization tasks and datasets, the current training paradigm for few-shot summarization systems ignores potentially shareable knowledge in heterogeneous datasets. To this end, we propose \textsc{UniSumm}, a unified few-shot summarization model pre-trained with multiple summarization tasks and can be prefix-tuned to excel at any few-shot summarization task. Meanwhile, to better evaluate few-shot summarizers, under the principles of diversity and robustness, we assemble and release a new benchmark \textsc{SummZoo}. It consists of $8$ summarization tasks with multiple sets of few-shot samples for each task, covering diverse domains. Experimental results and analysis show that \textsc{UniSumm} outperforms strong baselines by a large margin across all sub-tasks in \textsc{SummZoo} under both automatic and human evaluations and achieves comparable results in human evaluation compared with a GPT-3.5 model.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。