揭露裂痕：基于检索增强型大语言模型的机器翻译的脆弱性
Exposing the Cracks: Vulnerabilities of Retrieval-Augmented LLM-based Machine Translation
AI总结：
针对检索增强型大语言模型机器翻译（REAL-MT）在嘈杂检索语境下可靠性不明的问题，提出噪声合成框架与新指标评估其鲁棒性，发现低资源语言对及大型推理模型更易受噪声影响，揭示权衡并强调自验证集成机制的必要性。
摘要：
基于检索增强型大语言模型的机器翻译（REAL-MT）在习语翻译等知识密集型任务中展现出潜力，但在嘈杂检索语境下的可靠性仍未被充分理解，尽管这是现实部署中的常见挑战。为填补这一空白，我们提出了一个噪声合成框架和新指标，以系统评估REAL-MT的鲁棒性。利用该框架，我们用Qwen系列模型实例化REAL-MT，包括标准大语言模型和具有增强推理能力的大型推理模型（LRMs），并在合成噪声下评估它们在高、中、低资源语言对习语翻译任务上的性能。结果显示，更依赖检索语境的低资源语言对在噪声下的性能下降比高资源语言对更严重，且常产生无意义翻译。尽管LRMs具有增强的推理能力，但它们在纠错方面没有改进，甚至更易受噪声影响，倾向于将错误语境合理化。我们发现这源于注意力从源语言习语转移到噪声内容，同时尽管准确性下降，置信度却上升，表明校准不佳。为缓解这些问题，我们研究了无训练和微调策略，它们以清洁语境下的性能为代价提高了鲁棒性，揭示了根本权衡。我们的发现突显了当前方法的局限性，强调了对自验证集成机制的需求。
英文摘要：
\textbf{RE}trieval-\textbf{A}ugmented \textbf{L}LM-based \textbf{M}achine \textbf{T}ranslation (REAL-MT) shows promise for knowledge-intensive tasks like idiomatic translation, but its reliability under noisy retrieval contexts remains poorly understood despite this being a common challenge in real-world deployment. To address this gap, we propose a noise synthesis framework and new metrics to evaluate the robustness of REAL-MT systematically. Using this framework, we instantiate REAL-MT with Qwen-series models, including standard LLMs and large reasoning models (LRMs) with enhanced reasoning, and evaluate their performance on idiomatic translation across high-, medium-, and low-resource language pairs under synthesized noise. Our results show that low-resource language pairs, which rely more heavily on retrieved context, degrade more severely under noise than high-resource ones and often produce nonsensical translations. Although LRMs possess enhanced reasoning capabilities, they show no improvement in error correction and are even more susceptible to noise, tending to rationalize incorrect contexts. We find that this stems from an attention shift away from the source idiom to noisy content, while confidence increases despite declining accuracy, indicating poor calibration. To mitigate these issues, we investigate training-free and fine-tuning strategies, which improve robustness at the cost of performance in clean contexts, revealing a fundamental trade-off. Our findings highlight the limitations of current approaches, underscoring the need for self-verifying integration mechanisms.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
