Halluverse-M^3: 一个多任务多语言的幻觉基准测试用于LLMs中的幻觉
Halluverse-M^3: A multitask multilingual benchmark for hallucination in LLMs
AI总结：
Halluverse-M^3是一个多语言多任务基准测试，用于评估大型语言模型中的幻觉检测性能，涵盖四种语言和两种生成任务，揭示了不同任务和语言下的幻觉检测挑战。
摘要：
大型语言模型中的幻觉问题仍然是一个持续的挑战，特别是在多语言和生成性设置中，事实一致性难以维持。尽管最近的模型在以英语为中心的基准测试中表现强劲，但其在不同语言、任务和幻觉类型上的行为尚未完全了解。在本工作中，我们介绍了Halluverse-M^3，一个旨在系统分析多语言、多生成任务和多幻觉类别的幻觉数据集。Halluverse-M^3覆盖了四种语言：英语、阿拉伯语、印地语和土耳其语，并支持两种生成任务：问答和对话摘要。该数据集明确区分了实体级、关系级和句子级的幻觉。幻觉输出是通过受控的编辑过程构建的，并通过人类标注者验证，确保原始内容与幻觉生成之间有清晰的一致性。使用此数据集，我们评估了各种现代开源和专有语言模型在细粒度幻觉检测中的表现。我们的结果表明，问答任务比对话摘要任务更容易，而句子级幻觉即使对于最强的模型也仍然具有挑战性。表现最高的是英语，在低资源语言中逐渐下降，印地语显示出最低的检测准确性。总体而言，Halluverse-M^3为研究多语言、多任务设置中的幻觉问题提供了一个现实且具有挑战性的基准测试。我们发布此数据集以支持未来关于幻觉检测和缓解的研究。
英文摘要：
Hallucinations in large language models remain a persistent challenge, particularly in multilingual and generative settings where factual consistency is difficult to maintain. While recent models show strong performance on English-centric benchmarks, their behavior across languages, tasks, and hallucination types is not yet well understood. In this work, we introduce Halluverse-M^3, a dataset designed to enable systematic analysis of hallucinations across multiple languages, multiple generation tasks, and multiple hallucination categories. Halluverse-M^3 covers four languages, English, Arabic, Hindi, and Turkish, and supports two generation tasks: question answering and dialogue summarization. The dataset explicitly distinguishes between entity-level, relation-level, and sentence-level hallucinations. Hallucinated outputs are constructed through a controlled editing process and validated by human annotators, ensuring clear alignment between original content and hallucinated generations. Using this dataset, we evaluate a diverse set of contemporary open-source and proprietary language models on fine-grained hallucination detection. Our results show that question answering is consistently easier than dialogue summarization, while sentence-level hallucinations remain challenging even for the strongest models. Performance is highest in English and degrades in lower-resource languages, with Hindi exhibiting the lowest detection accuracy. Overall, Halluverse-M^3 provides a realistic and challenging benchmark for studying hallucinations in multilingual, multi-task settings. We release the dataset to support future research on hallucination detection and mitigation\footnote{https://huggingface.co/datasets/sabdalja/HalluVerse-M3}.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
