DiscoTrack：一个面向话语追踪的多语言大语言模型基准
DiscoTrack: A Multilingual LLM Benchmark for Discourse Tracking
AI总结：
DiscoTrack是一个覆盖12种语言、四个话语理解层级的LLM基准，用于评估跨句、跨段和跨说话人的隐式信息整合与语用推理能力，实验表明这些任务对当前最先进模型仍具挑战性。
摘要：
近期的大语言模型基准已在多种现象上测试了模型，但仍主要侧重于用于提取显式信息的自然语言理解，例如问答或摘要，其回答通常针对单个句子中的信息。我们仍然缺乏更具挑战性且重要的多语言基准，这些基准应关注话语追踪背景下跨较长文档的隐式信息和语用推理：即跨句子、段落和多个说话人话语整合与聚合信息。为此，我们提出了DiscoTrack，这是一个面向12种语言和四个话语理解层级（显著性识别、实体追踪、话语关系和桥接推理）的一系列任务的大语言模型基准。我们的评估表明，即使对于最先进的模型，这些任务仍然具有挑战性。
英文摘要：
Recent LLM benchmarks have tested models on a range of phenomena, but are still focused primarily on natural language understanding for extraction of explicit information, such as QA or summarization, with responses often targeting information from individual sentences. We are still lacking more challenging, and importantly also multilingual, benchmarks focusing on implicit information and pragmatic inferences across larger documents in the context of discourse tracking: integrating and aggregating information across sentences, paragraphs and multiple speaker utterances. To this end, we present DiscoTrack, an LLM benchmark targeting a range of tasks across 12 languages and four levels of discourse understanding: salience recognition, entity tracking, discourse relations and bridging inference. Our evaluation shows that these tasks remain challenging, even for state-of-the-art models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
