arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2307.04018cs.CL

重新审视跨语言摘要:基于语料库的研究与一个改进标注的新基准

Revisiting Cross-Lingual Summarization: A Corpus-based Study and A New Benchmark with Improved Annotation

Zhejiang University(浙江大学);Westlake University(西湖大学);UIUC(伊利诺伊大学厄巴纳-香槟分校);Sichuan Lan-bridge Information Technology Co., Ltd.(四川语言桥信息技术有限公司);Westlake Institute for Advanced Study(西湖高等研究院)

Yulong Chen, Huajian Zhang, Yijie Zhou, Xuefeng Bai, Yueguan Wang, Ming Zhong, Jianhao Yan, Yafu Li, Judy Li, Michael Zhu, Yue Zhang

AI总结:

本文针对现有跨语言摘要语料库因直接翻译预标注摘要而引入错误的问题,提出考虑源输入上下文的对话摘要基准ConvSumX及模拟人工标注的2-Step方法,实验证明其更忠实且性能超越强基线。

摘要:

现有的大多数跨语言摘要(CLS)工作通过将预先标注的摘要从一种语言简单直接地翻译到另一种语言来构建CLS语料库,这可能会同时包含摘要过程和翻译过程产生的错误。为了解决这一问题,我们提出了ConvSumX,一个跨语言对话摘要基准,采用了一种明确考虑源输入上下文的新标注模式。ConvSumX包含2个面向不同现实场景的子任务,每个子任务覆盖3种语言方向。我们对ConvSumX以及3个广泛使用的人工标注CLS语料库进行了深入分析,并通过实验发现ConvSumX对输入文本更加忠实。此外,基于相同的直觉,我们提出了一种2-Step方法,该方法同时以对话和摘要作为输入来模拟人工标注过程。实验结果表明,在自动评估和人工评估下,2-Step方法在ConvSumX上均超越了强基线。分析表明,源输入文本和摘要对于建模跨语言摘要都至关重要。

英文摘要:

Most existing cross-lingual summarization (CLS) work constructs CLS corpora by simply and directly translating pre-annotated summaries from one language to another, which can contain errors from both summarization and translation processes. To address this issue, we propose ConvSumX, a cross-lingual conversation summarization benchmark, through a new annotation schema that explicitly considers source input context. ConvSumX consists of 2 sub-tasks under different real-world scenarios, with each covering 3 language directions. We conduct thorough analysis on ConvSumX and 3 widely-used manually annotated CLS corpora and empirically find that ConvSumX is more faithful towards input text. Additionally, based on the same intuition, we propose a 2-Step method, which takes both conversation and summary as input to simulate human annotation process. Experimental results show that 2-Step method surpasses strong baselines on ConvSumX under both automatic and human evaluation. Analysis shows that both source input text and summary are crucial for modeling cross-lingual summaries.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑