挑战多语种大语言模型：用于揭示翻译中幻觉现象的新分类体系与基准
Challenging Multilingual LLMs: A New Taxonomy and Benchmark for Unraveling Hallucination in Translation
AI总结：
针对现有机器翻译基准难以暴露多语种大语言模型翻译幻觉的问题，研究提出区分指令脱离与源文脱离的分类框架，构建覆盖11个翻译方向的HalloMTBench基准，评估17个LLMs后发现多种幻觉触发因素，为翻译故障诊断提供测试平台。
摘要：
大语言模型（LLMs）推动了机器翻译的发展，但仍易受幻觉问题影响。遗憾的是，现有机器翻译基准无法暴露多语种LLMs的这类缺陷。为揭示多语种LLMs的翻译幻觉，我们提出一个诊断框架，其配套的分类体系将“指令脱离”与“源文脱离”两类幻觉区分开。在该分类体系指导下，我们构建了HalloMTBench——一个覆盖11个英转X翻译方向、经人工验证的多语种基准。我们使用4个前沿LLMs生成候选译文，通过LLM评审集成方法与专家验证对候选内容进行细致筛查，最终整理出5435个高质量样本。我们在HalloMTBench上评估了17个LLMs，结果显示存在 distinct 的“幻觉触发因素”——即反映模型规模、源文长度敏感度、语言偏见以及强化学习（RL）放大的语言混合现象的独特失效模式。HalloMTBench为诊断LLM翻译失效提供了具有前瞻性的测试平台，可通过https://huggingface.co/collections/AIDC-AI/marco-mt获取。
英文摘要：
Large Language Models (LLMs) have advanced machine translation but remain vulnerable to hallucinations. Unfortunately, existing MT benchmarks are not capable of exposing failures in multilingual LLMs. To disclose hallucination in multilingual LLMs, we introduce a diagnostic framework with a taxonomy that separates Instruction Detachment from Source Detachment. Guided by this taxonomy, we create HalloMTBench, a multilingual, human-verified benchmark across 11 English-to-X directions. We employed 4 frontier LLMs to generate candidates and scrutinize these candidates with an ensemble of LLM judges, and expert validation. In this way, we curate 5,435 high-quality instances. We have evaluated 17 LLMs on HalloMTBench. Results reveal distinct ``hallucination triggers'' -- unique failure patterns reflecting model scale, source length sensitivity, linguistic biases, and Reinforcement-Learning (RL) amplified language mixing. HalloMTBench offers a forward-looking testbed for diagnosing LLM translation failures. HalloMTBench is available in https://huggingface.co/collections/AIDC-AI/marco-mt.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
