arXiv 2504.206992025-04-29cs.CLcs.AI

LLM能否检测复述和机器翻译中的内在幻觉？

Can LLMs Detect Intrinsic Hallucinations in Paraphrasing and Machine Translation?

RISE Research Institutes of Sweden, Department of Computer Science（瑞典RISE研究机构，计算机科学系）；Uppsala University, Department of Linguistics and Philology（乌普萨拉大学，语言学与哲学系）；KTH Royal Institute of Technology, Division of Software and Computer Systems（皇家理工学院，软件与计算机系统系）；University of Edinburgh, School of Informatics（爱丁堡大学，信息学院）

Evangelia Gogoulou, Shorouq Zahra, Liane Guillou, Luise Dürlich, Joakim Nivre

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

基于HalluciGen任务评估开源LLM在翻译和复述中检测内在幻觉的能力，发现性能因模型而异但不受提示影响，且NLI模型表现相当，表明LLM并非唯一可行选择。

摘要：

LLM常见的一个问题是倾向于生成无意义、不合逻辑或事实上不正确的输出，通常被统称为幻觉。基于最近提出的用于幻觉检测和生成的HalluciGen任务，我们评估了一组开源LLM在两种条件生成任务（翻译和复述）中检测内在幻觉的能力。我们研究了模型性能在不同任务和语言间的差异，并考察了模型规模、指令微调及提示选择的影响。我们发现性能因模型而异，但在不同提示下保持一致。最后，我们发现NLI模型表现相当，表明基于LLM的检测器并非该特定任务的唯一可行选择。

英文摘要：

A frequently observed problem with LLMs is their tendency to generate output that is nonsensical, illogical, or factually incorrect, often referred to broadly as hallucination. Building on the recently proposed HalluciGen task for hallucination detection and generation, we evaluate a suite of open-access LLMs on their ability to detect intrinsic hallucinations in two conditional generation tasks: translation and paraphrasing. We study how model performance varies across tasks and language and we investigate the impact of model size, instruction tuning, and prompt choice. We find that performance varies across models but is consistent across prompts. Finally, we find that NLI models perform comparably well, suggesting that LLM-based detectors are not the only viable option for this specific task.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。