大型语言模型的幻觉检测器是否受到低资源效应影响？
Do LLM hallucination detectors suffer from low-resource effect?
AI总结：
研究发现，幻觉检测器在低资源语言中表现较任务本身更稳健，可能因内部机制编码了不确定性信号。
摘要：
尽管大型语言模型（LLMs）在广泛的任务中表现优于人类，但仍可能以意想不到的方式失败。我们关注两种普遍的失败模式：（i）幻觉，即模型产生关于现实的错误信息；（ii）低资源效应，即模型在高资源语言如英语中表现出色，但在低资源语言如孟加拉语中性能显著下降。我们研究这两种问题的交集，并提出问题：幻觉检测器是否受到低资源效应的影响？我们在三个领域（事实回忆、STEM和人文学科）的五个任务上进行了实验。使用四个LLMs和三个幻觉检测器进行实验，发现一个有趣的结果：正如预期的那样，低资源语言中的任务准确率相比英语有大幅下降。然而，检测器的准确率下降通常比任务准确率下降小几倍。我们的发现表明，即使在低资源语言中，LLMs的内部机制可能也编码了关于不确定性的信号。此外，检测器在语言内部（即使是非英语）和多语言设置中表现稳健，但在没有语言内部监督的跨语言设置中并不稳健。
英文摘要：
LLMs, while outperforming humans in a wide range of tasks, can still fail in unanticipated ways. We focus on two pervasive failure modes: (i) hallucinations, where models produce incorrect information about the world, and (ii) the low-resource effect, where the models show impressive performance in high-resource languages like English but the performance degrades significantly in low-resource languages like Bengali. We study the intersection of these issues and ask: do hallucination detectors suffer from the low-resource effect? We conduct experiments on five tasks across three domains (factual recall, STEM, and Humanities). Experiments with four LLMs and three hallucination detectors reveal a curious finding: As expected, the task accuracies in low-resource languages experience large drops (compared to English). However, the drop in detectors' accuracy is often several times smaller than the drop in task accuracy. Our findings suggest that even in low-resource languages, the internal mechanisms of LLMs might encode signals about their uncertainty. Further, the detectors are robust within language (even for non-English) and in multilingual setups, but not in cross-lingual settings without in-language supervision.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
