LLMs在不同语言中产生幻觉的程度有多大？对多语言现实估计LLM幻觉的探讨
How Much Do LLMs Hallucinate across Languages? On Realistic Multilingual Estimation of LLM Hallucination
AI总结：
研究评估了多语言LLM在长形式问答中的幻觉程度，发现高资源语言中幻觉率与语言规模无关，且支持更多语言的LLM幻觉率更高。
摘要：
在信息失真的时代，幻觉——大型语言模型（LLMs）倾向于生成非事实或不忠实的响应——代表了其全球应用的主要风险。尽管LLMs变得越来越多语言化，但大多数关于检测和量化LLM幻觉的研究都是（a）以英语为中心的，（b）专注于机器翻译（MT）和摘要，这些任务在现实场景中不如开放信息查询常见。相比之下，我们旨在量化在知识密集型长形式问答（LFQA）中不同语言的LLM幻觉程度。为此，我们训练了一个多语言幻觉检测模型，并在30种语言和6个开源LLM家族上进行了大规模研究。我们从一个英语幻觉检测数据集开始，并依靠MT来翻译训练检测模型。我们还手动标注了五种高资源语言的黄金数据；然后我们证明，对于这些语言，银测试集（LLM生成）和黄金测试集的幻觉率估计是相似的，这验证了使用银数据来估计其他语言的幻觉率的可行性。对于最终的幻觉率估计，我们构建了30种语言的开放域问答数据集，使用LLM生成的提示和维基百科文章作为参考。我们的分析显示，LLMs在绝对意义上，由于更长的响应，在高资源语言中会产生更多token的幻觉，但实际幻觉率（即归一化长度）似乎与语言数字足迹的大小无关。我们还发现，较小的LLM会产生更多幻觉，并且显著的是，具有更广泛语言支持的LLM显示出更高的幻觉率。
英文摘要：
In the age of misinformation, hallucination - the tendency of Large Language Models (LLMs) to generate non-factual or unfaithful responses - represents the main risk for their global utility. Despite LLMs becoming increasingly multilingual, the vast majority of research on detecting and quantifying LLM hallucination are (a) English-centric and (b) focus on machine translation (MT) and summarization, tasks that are less common in realistic settings than open information seeking. In contrast, we aim to quantify the extent of LLM hallucination across languages in knowledge-intensive long-form question answering (LFQA). To this end, we train a multilingual hallucination detection model and conduct a large-scale study across 30 languages and 6 open-source LLM families. We start from an English hallucination detection dataset and rely on MT to translate-train a detection model. We also manually annotate gold data for five high-resource languages; we then demonstrate, for these languages, that the estimates of hallucination rates are similar between silver (LLM-generated) and gold test sets, validating the use of silver data for estimating hallucination rates for other languages. For the final rates estimation, we build open-domain QA dataset for 30 languages with LLM-generated prompts and Wikipedia articles as references. Our analysis shows that LLMs, in absolute terms, hallucinate more tokens in high-resource languages due to longer responses, but that the actual hallucination rates (i.e., normalized for length) seems uncorrelated with the sizes of languages' digital footprints. We also find that smaller LLMs hallucinate more, and significantly, LLMs with broader language support display higher hallucination rates.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
