AMTSS：一种用于多语言语言推理的自适应多教师单学生知识蒸馏框架
AMTSS: An Adaptive Multi-Teacher Single-Student Knowledge Distillation Framework For Multilingual Language Inference
AI总结：
提出AMTSS自适应多教师单学生知识蒸馏框架，通过自适应学习策略和共享编码器，在多语言推理任务中实现高效知识迁移，并在XNLI和阿里电商数据集上取得有竞争力的结果。
摘要：
知识蒸馏对于将多语言预训练语言模型投入实际应用至关重要。为了在多语言环境中支持具有成本效益的语言推理，我们提出了AMTSS，一种自适应多教师单学生蒸馏框架，该框架允许将知识从多个教师蒸馏到一个学生。我们首先引入了一种自适应学习策略和教师重要性权重，使学生能够有效地从最大间隔教师中学习，并轻松适应新语言。此外，我们提出了一个共享的学生编码器，并配备不同的投影层以支持多种语言，这有助于大幅降低开发和机器成本。实验结果表明，AMTSS在公开的XNLI数据集和电商场景下的真实工业数据集AliExpress（AE）上取得了有竞争力的结果。
英文摘要：
Knowledge distillation is of key importance to launching multilingual pre-trained language models for real applications. To support cost-effective language inference in multilingual settings, we propose AMTSS, an adaptive multi-teacher single-student distillation framework, which allows distilling knowledge from multiple teachers to a single student. We first introduce an adaptive learning strategy and teacher importance weight, which enables a student to effectively learn from max-margin teachers and easily adapt to new languages. Moreover, we present a shared student encoder with different projection layers in support of multiple languages, which contributes to largely reducing development and machine cost. Experimental results show that AMTSS gains competitive results on the public XNLI dataset and the realistic industrial dataset AliExpress (AE) in the E-commerce scenario.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
