MT-PATCHER：面向机器翻译的大语言模型选择性可扩展知识蒸馏
MT-PATCHER: Selective and Extendable Knowledge Distillation from Large Language Models for Machine Translation
AI总结：
本文提出MT-Patcher框架，通过选择性纠正学生翻译错误并利用LLM合成多样化上下文和潜在错误，以约10%样本微调即可达到传统知识蒸馏效果，并提升模型在未见上下文和词语上的泛化能力。
摘要：
大语言模型（LLM）已在机器翻译（MT）领域展现出强大能力，但其计算成本和延迟较高。因此，将巨型LLM中的翻译知识迁移到中等规模的机器翻译模型是一个有前景的研究方向。然而，传统的知识蒸馏方法未考虑学生模型和教师模型的能力差异，因而反复向学生模型传授其已学过的知识，且无法扩展到新的上下文和知识。本文提出一个名为MT-Patcher的框架，以选择性、全面且主动的方式将LLM的知识迁移到现有MT模型中。考虑到学生MT模型当前的翻译能力，我们仅识别并纠正其翻译错误，而不是从教师模型蒸馏整个翻译结果。利用LLM强大的语言能力，我们指导LLM教师合成多样化的上下文，并预测学生模型更多潜在错误。在特定语言现象翻译和通用MT基准上的实验结果表明，在约10%的样本上微调学生MT模型即可取得与传统知识蒸馏方法相当的结果，而合成的潜在错误和多样化上下文进一步提升了模型在未见上下文和词语上的翻译性能。
英文摘要：
Large Language Models (LLM) have demonstrated their strong ability in the field of machine translation (MT), yet they suffer from high computational cost and latency. Therefore, transferring translation knowledge from giant LLMs to medium-sized machine translation models is a promising research direction. However, traditional knowledge distillation methods do not take the capability of student and teacher models into consideration, therefore repeatedly teaching student models on the knowledge they have learned, and failing to extend to novel contexts and knowledge. In this paper, we propose a framework called MT-Patcher, which transfers knowledge from LLMs to existing MT models in a selective, comprehensive and proactive manner. Considering the current translation ability of student MT models, we only identify and correct their translation errors, instead of distilling the whole translation from the teacher. Leveraging the strong language abilities of LLMs, we instruct LLM teachers to synthesize diverse contexts and anticipate more potential errors for the student. Experiment results on translating both specific language phenomena and general MT benchmarks demonstrate that finetuning the student MT model on about 10% examples can achieve comparable results to the traditional knowledge distillation method, and synthesized potential errors and diverse contexts further improve translation performances on unseen contexts and words.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
