使用聚焦幻觉的偏好优化缓解大型语言模型中的幻觉翻译
Mitigating Hallucinated Translations in Large Language Models with Hallucination-focused Preference Optimization
AI总结：
针对LLM翻译幻觉问题，提出在训练阶段内在缓解幻觉的数据创建框架，通过生成聚焦幻觉的偏好数据集微调模型，使幻觉率在五个语言对上平均降低96%，零样本设置下平均降低89%，且保持翻译质量。
摘要：
机器翻译（MT）正在经历范式转变，基于微调大型语言模型（LLM）的系统正变得日益具有竞争力，能够与专门针对翻译任务训练的传统编码器-解码器模型相抗衡。然而，基于LLM的系统面临更高的生成幻觉风险，这可能严重破坏用户的信任与安全。以往关于幻觉缓解的研究大多集中于传统MT模型，其解决方案涉及事后缓解——检测幻觉翻译并对其进行重新翻译。尽管有效，但这种方法在生产环境中引入了部署额外工具的复杂性，并增加了延迟。为了解决这些局限性，我们提出了一种在模型训练阶段内在地学习缓解幻觉的方法。具体而言，我们引入了一个数据创建框架，用于生成聚焦幻觉的偏好数据集。在这些偏好数据集上对LLM进行微调，使五个语言对的幻觉率平均降低了96%，同时保持了整体翻译质量。在零样本设置下，我们的方法在三个未见过的目标语言上平均减少了89%的幻觉。
英文摘要：
Machine Translation (MT) is undergoing a paradigm shift, with systems based on fine-tuned large language models (LLM) becoming increasingly competitive with traditional encoder-decoder models trained specifically for translation tasks. However, LLM-based systems are at a higher risk of generating hallucinations, which can severely undermine user's trust and safety. Most prior research on hallucination mitigation focuses on traditional MT models, with solutions that involve post-hoc mitigation - detecting hallucinated translations and re-translating them. While effective, this approach introduces additional complexity in deploying extra tools in production and also increases latency. To address these limitations, we propose a method that intrinsically learns to mitigate hallucinations during the model training phase. Specifically, we introduce a data creation framework to generate hallucination focused preference datasets. Fine-tuning LLMs on these preference datasets reduces the hallucination rate by an average of 96% across five language pairs, while preserving overall translation quality. In a zero-shot setting our approach reduces hallucinations by 89% on an average across three unseen target languages.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
