比较多语言生成中的幻觉检测指标
Comparing Hallucination Detection Metrics for Multilingual Generation
AI总结：
本文评估 ROUGE、命名实体重叠和 NLI 类指标在跨语言传记摘要中的幻觉检测效果，发现 NLI 指标整体优于词汇指标和监督模型，但在单事实幻觉与低资源语言上仍不足。
摘要：
尽管许多幻觉检测技术已在英文文本上得到评估，但它们在多语言语境中的有效性仍不清楚。本文评估了各种事实性幻觉检测指标（如 ROUGE 和命名实体重叠等词汇指标，以及基于自然语言推理（NLI）的指标）在跨语言生成的传记摘要中识别幻觉的能力。我们比较了自动指标之间的相关性，以及它们是否与人类对事实性的判断一致。分析表明，虽然词汇指标效果不佳，但基于 NLI 的指标表现良好，在许多设置中与人工标注相关，并且往往优于监督模型。然而，NLI 指标仍然存在局限，因为它们不能很好地检测单一事实幻觉，并且在低资源语言上会失效。因此，我们的发现凸显了现有非英语语言幻觉检测方法的差距，并推动未来研究为 LLM 幻觉开发更稳健的多语言检测方法。
英文摘要：
While many hallucination detection techniques have been evaluated on English text, their effectiveness in multilingual contexts remains unknown. This paper assesses how well various factual hallucination detection metrics (lexical metrics like ROUGE and Named Entity Overlap, and Natural Language Inference (NLI)-based metrics) identify hallucinations in generated biographical summaries across languages. We compare how well automatic metrics correlate to each other and whether they agree with human judgments of factuality. Our analysis reveals that while the lexical metrics are ineffective, NLI-based metrics perform well, correlating with human annotations in many settings and often outperforming supervised models. However, NLI metrics are still limited, as they do not detect single-fact hallucinations well and fail for lower-resource languages. Therefore, our findings highlight the gaps in exisiting hallucination detection methods for non-English languages and motivate future research to develop more robust multilingual detection methods for LLM hallucinations.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
