通过两阶段对比学习改进词语翻译
Improving Word Translation via Two-Stage Contrastive Learning
AI总结：
本文提出两阶段对比学习框架用于双语词典归纳，先精化静态词嵌入的跨语言映射，再对 mBERT 进行面向 BLI 的对比微调，在 28 个语言对的 112 个设置上均取得提升。
摘要：
词语翻译或双语词典归纳（BLI）是一项关键的跨语言任务，旨在弥合不同语言之间的词汇鸿沟。在这项工作中，我们针对 BLI 任务提出了一个稳健且有效的两阶段对比学习框架。在阶段 C1 中，我们提出通过对比学习目标来精化静态词嵌入（WEs）之间的标准跨语言线性映射；我们还展示了如何将其整合到自学习过程中，以获得更精化的跨语言映射。在阶段 C2 中，我们对 mBERT 进行面向 BLI 的对比微调，释放其词语翻译能力。我们还表明，从“经过 C2 微调的”mBERT 中导出的静态词嵌入能够补充阶段 C1 的静态词嵌入。在多种语言和不同实验设置下的标准 BLI 数据集上进行的综合实验表明，我们的框架取得了显著提升。虽然阶段 C1 的 BLI 方法已经在我们比较的所有最先进 BLI 方法之上取得了显著提升，但完整的两阶段框架带来了更强的改进：例如，我们报告了 112/112 个 BLI 设置上的提升，涵盖 28 个语言对。
英文摘要：
Word translation or bilingual lexicon induction (BLI) is a key cross-lingual task, aiming to bridge the lexical gap between different languages. In this work, we propose a robust and effective two-stage contrastive learning framework for the BLI task. At Stage C1, we propose to refine standard cross-lingual linear maps between static word embeddings (WEs) via a contrastive learning objective; we also show how to integrate it into the self-learning procedure for even more refined cross-lingual maps. In Stage C2, we conduct BLI-oriented contrastive fine-tuning of mBERT, unlocking its word translation capability. We also show that static WEs induced from the `C2-tuned' mBERT complement static WEs from Stage C1. Comprehensive experiments on standard BLI datasets for diverse languages and different experimental setups demonstrate substantial gains achieved by our framework. While the BLI method from Stage C1 already yields substantial gains over all state-of-the-art BLI methods in our comparison, even stronger improvements are met with the full two-stage framework: e.g., we report gains for 112/112 BLI setups, spanning 28 language pairs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
