arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2406.07036cs.CLcs.AI
收藏

更加关注源上下文:缓解大型语言模型产生的不忠实翻译

Paying More Attention to Source Context: Mitigating Unfaithful Translations from Large Language Model

Institute of Computing and Intelligence, Harbin Institute of Technology, Shenzhen(哈尔滨工业大学(深圳)计算与智能学院);Peng Cheng Laboratory, Shenzhen(鹏城实验室)

Hongbin Zhang, Kehai Chen, Xuefeng Bai, Yang Xiang, Min Zhang

AI总结:

针对仅解码器 LLM 因偏向目标前缀而产生不忠实翻译的问题,本文从零样本提示的源/目标注意力调控与指令微调三方面增强源上下文关注,并在多语言测试中验证其可减少幻觉翻译。

摘要:

大型语言模型(LLM)已经展现出令人瞩目的多语言机器翻译能力。然而,与编码器-解码器风格的模型不同,仅解码器 LLM 在源上下文与目标上下文之间缺乏显式对齐。分析生成过程中的贡献分数表明,LLM 可能偏向先前生成的 token,而不是相应的源 token,从而导致不忠实翻译。为解决这一问题,我们提出在零样本提示中从源和目标两个视角鼓励 LLM 更加关注源上下文:1)调整源上下文注意力权重;2)抑制无关目标前缀的影响;此外,我们提出 3)在指令微调中避免过度依赖目标前缀。来自人工收集的、聚焦 LLM 生成不忠实翻译的不忠实性测试集以及通用测试集的实验结果,验证了我们的方法在多个语言对上的有效性。进一步的人工评估表明,我们的方法能够有效减少幻觉翻译并促进忠实翻译的生成。

英文摘要:

Large language models (LLMs) have showcased impressive multilingual machine translation ability. However, unlike encoder-decoder style models, decoder-only LLMs lack an explicit alignment between source and target contexts. Analyzing contribution scores during generation processes revealed that LLMs can be biased towards previously generated tokens over corresponding source tokens, leading to unfaithful translations. To address this issue, we propose to encourage LLMs to pay more attention to the source context from both source and target perspectives in zeroshot prompting: 1) adjust source context attention weights; 2) suppress irrelevant target prefix influence; Additionally, we propose 3) avoiding over-reliance on the target prefix in instruction tuning. Experimental results from both human-collected unfaithfulness test sets focusing on LLM-generated unfaithful translations and general test sets, verify our methods' effectiveness across multiple language pairs. Further human evaluation shows our method's efficacy in reducing hallucinatory translations and facilitating faithful translation generation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑