arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.01535cs.CLcs.AI
收藏

论辩摘要中多样性的实证分析

An Empirical Analysis of Diversity in Argument Summarization

Leiden University(莱顿大学);Vrije Universiteit Amsterdam(阿姆斯特丹自由大学);TU Delft(代尔夫特理工大学)

Michiel van der Meer, Piek Vossen, Catholijn M. Jonker, Pradeep K. Murukannaiah

AI总结:

本文通过引入观点、标注者和来源三方面多样性,实证分析了 Key Point Analysis 论辩摘要任务,发现现有模型在少数论点、多源数据和主观性对齐上存在困难,但通用 LLM 与专用 KPA 模型互补,且数据多样化可改善泛化。

摘要:

呈现高层级论点是促进在线社会讨论参与的关键任务。当前的论辩摘要方法忽略了该任务的一个重要方面——捕捉多样性,这对容纳多种观点很重要。我们引入了多样性的三个方面:观点、标注者和来源。我们评估了一种名为 Key Point Analysis 的流行论辩摘要任务的方法,结果表明这些方法在以下方面存在困难:(1) 代表少数人共享的论点,(2) 处理来自不同来源的数据,以及 (3) 与人工标注中的主观性保持一致。我们发现,通用 LLM 和专用 KPA 模型均表现出此行为,但具有互补优势。此外,我们观察到训练数据的多样化可能改善泛化能力。解决论辩摘要中的多样性需要结合多种策略来应对主观性。

英文摘要:

Presenting high-level arguments is a crucial task for fostering participation in online societal discussions. Current argument summarization approaches miss an important facet of this task -- capturing diversity -- which is important for accommodating multiple perspectives. We introduce three aspects of diversity: those of opinions, annotators, and sources. We evaluate approaches to a popular argument summarization task called Key Point Analysis, which shows how these approaches struggle to (1) represent arguments shared by few people, (2) deal with data from various sources, and (3) align with subjectivity in human-provided annotations. We find that both general-purpose LLMs and dedicated KPA models exhibit this behavior, but have complementary strengths. Further, we observe that diversification of training data may ameliorate generalization. Addressing diversity in argument summarization requires a mix of strategies to deal with subjectivity.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑