CADS:抽象式对话摘要挑战的系统文献综述
CADS: A Systematic Literature Review on the Challenges of Abstractive Dialogue Summarization
AI总结:
本文系统综述2019-2024年间1262篇论文,梳理抽象式对话摘要的语言、结构、理解等六类挑战及其技术关联,指出理解、事实性等难题仍待攻克,并探讨了大语言模型的影响。
摘要:
抽象式对话摘要任务是将对话提炼为信息丰富且简洁的摘要。尽管已有关于该主题的综述,但缺乏全面阐述对话摘要挑战、统一对该任务的不同理解,并将所提出的技术、数据集和评估指标与挑战对齐的工作。本文通过系统性地回顾2019年至2024年间发表的1262篇独特研究论文,基于Semantic Scholar和DBLP数据库,总结了基于Transformer的英语对话抽象式摘要研究。我们涵盖了对话摘要中存在的主要挑战(即语言、结构、理解、说话者、显著性和事实性),并将其与相应技术联系起来,如图基方法、额外训练任务和规划策略,这些技术通常过度依赖基于BART的编码器-解码器模型。我们发现,虽然某些挑战(如语言)由于训练方法已取得显著进展,但其他挑战(如理解、事实性和显著性)仍然困难重重,并蕴含着重要的研究机会。我们调查了这些方法通常如何被评估,涵盖了对话子领域(如会议、医疗)的数据集、用于评估分数和标注者一致性的既定自动指标和人工评估方法。我们观察到,只有少数数据集跨越所有子领域。ROUGE指标使用最为广泛,而人工评估在报告时常常缺乏关于标注者内部一致性和标注指南的足够细节。此外,我们讨论了近期探索的大语言模型可能带来的影响,并得出结论:尽管相关性和难度可能发生转变,我们所描述的挑战分类法仍然具有相关性。
英文摘要:
Abstractive dialogue summarization is the task of distilling conversations into informative and concise summaries. Although reviews have been conducted on this topic, there is a lack of comprehensive work detailing the challenges of dialogue summarization, unifying the differing understanding of the task, and aligning proposed techniques, datasets, and evaluation metrics with the challenges. This article summarizes the research on Transformer-based abstractive summarization for English dialogues by systematically reviewing 1262 unique research papers published between 2019 and 2024, relying on the Semantic Scholar and DBLP databases. We cover the main challenges present in dialog summarization (i.e., language, structure, comprehension, speaker, salience, and factuality) and link them to corresponding techniques such as graph-based approaches, additional training tasks, and planning strategies, which typically overly rely on BART-based encoder-decoder models. We find that while some challenges, like language, have seen considerable progress, mainly due to training methods, others, such as comprehension, factuality, and salience, remain difficult and hold significant research opportunities. We investigate how these approaches are typically assessed, covering the datasets for the subdomains of dialogue (e.g., meeting, medical), the established automatic metrics and human evaluation approaches for assessing scores and annotator agreement. We observe that only a few datasets span across all subdomains. The ROUGE metric is the most used, while human evaluation is frequently reported without sufficient detail on inner-annotator agreement and annotation guidelines. Additionally, we discuss the possible implications of the recently explored large language models and conclude that despite a potential shift in relevance and difficulty, our described challenge taxonomy remains relevant.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。