arXiv 2402.006202024-02-01cs.CL

语篇中的行动者识别：大语言模型（LLM）的一项挑战？

Actor Identification in Discourse: A Challenge for LLMs?

University of Stuttgart（斯图加特大学）；University of Zagreb（萨格勒布大学）

Ana Barić, Sean Papay, Sebastian Padó

备注：: Proceedings of the EACL 2024 workshop on Computational Models of Discourse (St. Julian's, Malta)

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对语篇中政治行动者识别需还原代词对应规范名称的难题，对比传统NLP流水线与LLM，发现LLM指称识别准但规范形式生成差，提出结合分类器的混合模型，性能显著优于两类初始模型。

摘要：

识别在公共辩论中提出主张的政治行动者是构建语篇网络的关键步骤，语篇网络有助于分析社会辩论。然而，行动者识别颇具挑战性：通常，局部提到的主张提出者只是一个代词（如“他提出了[主张]”），因此还原规范的行动者名称需要语篇理解能力。我们将由专用自然语言处理（NLP）组件构成的传统流水线（与共指相关任务所用的组件类似）与大语言模型（LLM）进行对比，后者似乎很适合这一生成式任务。在一份包含德语报纸报道中行动者的语料库上开展评估后，我们意外发现LLM的表现更差。进一步分析表明，LLM非常擅长识别正确的指称对象，但难以生成正确的规范形式。这揭示出LLM在控制生成输出方面存在一个深层问题。实际上，将LLM与一个用于规范化其输出的分类器相结合的混合模型，表现大幅优于两种初始模型。

英文摘要：

The identification of political actors who put forward claims in public debate is a crucial step in the construction of discourse networks, which are helpful to analyze societal debates. Actor identification is, however, rather challenging: Often, the locally mentioned speaker of a claim is only a pronoun ("He proposed that [claim]"), so recovering the canonical actor name requires discourse understanding. We compare a traditional pipeline of dedicated NLP components (similar to those applied to the related task of coreference) with a LLM, which appears a good match for this generation task. Evaluating on a corpus of German actors in newspaper reports, we find surprisingly that the LLM performs worse. Further analysis reveals that the LLM is very good at identifying the right reference, but struggles to generate the correct canonical form. This points to an underlying issue in LLMs with controlling generated output. Indeed, a hybrid model combining the LLM with a classifier to normalize its output substantially outperforms both initial models.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

AI总结：

摘要：

英文摘要：

相关论文