arXiv 2504.119752025-04-28cs.CL

SemEval-2025 Task 3: Mu-SHROOM，关于幻觉及相关可观察过度生成错误的多语言共享任务

SemEval-2025 Task 3: Mu-SHROOM, the Multilingual Shared Task on Hallucinations and Related Observable Overgeneration Mistakes

University of Helsinki（赫尔辛基大学）；SiLO；Université de Lorraine & ICANS Strasbourg（洛林大学及斯特拉斯堡ICANS）；Université Bretagne Sud（布列塔尼南方大学）；CIMAT A. C.；University of Milano-Bicocca（米兰-布雷拉大学）；TU Darmstadt（德累斯顿技术大学）；Aveni；Charles University（查尔斯大学）；HiTZ Basque Center for Language Technology - Ixa（巴斯克语言技术中心-ixa）

Raúl Vázquez, Timothee Mickus, Elaine Zosa, Teemu Vahtola, Jörg Tiedemann, Aman Sinha, Vincent Segonne, Fernando Sánchez-Vega, Alessandro Raganato, Jindřich Lib… 展开作者

Raúl Vázquez, Timothee Mickus, Elaine Zosa, Teemu Vahtola, Jörg Tiedemann, Aman Sinha, Vincent Segonne, Fernando Sánchez-Vega, Alessandro Raganato, Jindřich Libovický, Jussi Karlgren, Shaoxiong Ji, Jindřich Helcl, Liane Guillou, Ona de Gibert, Jaione Bengoetxea, Joseph Attieh, Marianna Apidianaki

备注：: Mu-SHROOM is part of SemEval-2025 (Task 3). TBP: Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025)

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出Mu-SHROOM共享任务，将14种语言的LLM幻觉检测构建为跨度标注任务，收到43个团队的2618份提交，通过实证分析揭示了影响性能的关键因素及跨语言幻觉差异等挑战。

摘要：

我们提出了Mu-SHROOM共享任务，专注于检测指令微调大型语言模型（LLMs）输出中的幻觉及其他过度生成错误。Mu-SHROOM针对14种语言的通用LLMs，并将幻觉检测问题构建为跨度标注任务。我们收到了来自43个参赛团队的2,618份提交，这些团队采用了多种方法。大量提交凸显了社区对幻觉检测的兴趣。我们展示了参与系统的结果，并进行了实证分析以识别促成该任务强大性能的关键因素。我们还强调了当前的相关挑战，特别是不同语言间幻觉程度的差异以及标注幻觉跨度时的高标注者分歧。

英文摘要：

We present the Mu-SHROOM shared task which is focused on detecting hallucinations and other overgeneration mistakes in the output of instruction-tuned large language models (LLMs). Mu-SHROOM addresses general-purpose LLMs in 14 languages, and frames the hallucination detection problem as a span-labeling task. We received 2,618 submissions from 43 participating teams employing diverse methodologies. The large number of submissions underscores the interest of the community in hallucination detection. We present the results of the participating systems and conduct an empirical analysis to identify key factors contributing to strong performance in this task. We also emphasize relevant current challenges, notably the varying degree of hallucinations across languages and the high annotator disagreement when labeling hallucination spans.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。