用于零样本上下文机器翻译的 Anti-LM 解码
Anti-LM Decoding for Zero-shot In-context Machine Translation
AI总结：
针对零样本上下文机器翻译中 LLM 校准不足的问题，提出带衰减因子的 Anti-Language Model 对比解码目标，并在多种模型、语言方向和解码方式下取得最高 20 BLEU 的提升。
摘要：
零样本上下文学习是指模型只需获得指令即可执行任务的现象。然而，众所周知，预训练大语言模型在此任务上的校准较差。处理这种偏差最有效的方法之一是采用对比解码目标，该目标通过以某些上下文为条件来解释生成下一个 token 的先验概率。本工作提出了一种带有衰减因子的 Anti-Language Model 目标，旨在解决上下文机器翻译的弱点。我们在 3 种模型类型和规模、3 个语言方向，以及贪心解码和束搜索（B=5）上进行了实验。所提出的方法优于其他最先进的解码目标，在某些设置下观察到相较默认目标最高可达 20 个 BLEU 点的提升。
英文摘要：
Zero-shot In-context learning is the phenomenon where models can perform the task simply given the instructions. However, pre-trained large language models are known to be poorly calibrated for this task. One of the most effective approaches to handling this bias is to adopt a contrastive decoding objective, which accounts for the prior probability of generating the next token by conditioning on some context. This work introduces an Anti-Language Model objective with a decay factor designed to address the weaknesses of In-context Machine Translation. We conduct our experiments across 3 model types and sizes, 3 language directions, and for both greedy decoding and beam search ($B=5$). The proposed method outperforms other state-of-art decoding objectives, with up to $20$ BLEU point improvement from the default objective observed in some settings.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
