通过自知识蒸馏促进少资源场景下的泛化跨语言问答
Promoting Generalized Cross-lingual Question Answering in Few-resource Scenarios via Self-knowledge Distillation
AI总结：
本文面向少资源及零样本场景，提出结合跨语言采样与逐代自蒸馏的G-XLT方法，并用mAP@k动态调节蒸馏损失，显著提升跨语言抽取式QA性能。
摘要：
尽管多语言抽取式问答（QA）已取得实质性进展，但要让模型在各语言上保持较高且均匀分布的性能仍具挑战，尤其是对于资源有限的语言。我们研究跨语言迁移，主要关注泛化跨语言迁移（G-XLT）任务，其中问题语言与上下文语言不同——这一挑战迄今受到的关注仍然有限。我们的方法旨在利用一个在大规模数据集上训练的高性能多语言模型，并辅以数千个跨语言对齐的QA样例，来增强跨语言QA迁移。我们提出的策略将跨语言采样与先进的逐代自蒸馏训练相结合，以应对上述挑战。值得注意的是，我们引入了新颖的mAP@k系数来微调自知识蒸馏损失，从而动态调节教师模型的知识，以实现平衡且有效的知识迁移。我们对方法进行了广泛评估，以考察抽取式QA中的XLT和G-XLT能力。结果表明，我们的自知识蒸馏方法显著优于标准交叉熵微调。重要的是，与利用大量机器翻译数据的强基线相比，尽管在资源受限环境（甚至零样本场景）中运行面临相当大的挑战，我们的方法仍展现出具有竞争力的结果。除性能提升外，我们还通过全面分析和消融研究提供了有价值的见解，进一步证实了该方法的优势与局限。本质上，我们提出了一种实用解决方案，通过高效利用少量数据资源来改进跨语言QA迁移。
英文摘要：
Despite substantial progress in multilingual extractive Question Answering (QA), models with high and uniformly distributed performance across languages remain challenging, especially for languages with limited resources. We study cross-lingual transfer mainly focusing on the Generalized Cross-Lingual Transfer (G-XLT) task, where the question language differs from the context language - a challenge that has received limited attention thus far. Our approach seeks to enhance cross-lingual QA transfer using a high-performing multilingual model trained on a large-scale dataset, complemented by a few thousand aligned QA examples across languages. Our proposed strategy combines cross-lingual sampling and advanced self-distillation training in generations to tackle the previous challenge. Notably, we introduce the novel mAP@k coefficients to fine-tune self-knowledge distillation loss, dynamically regulating the teacher's model knowledge to perform a balanced and effective knowledge transfer. We extensively evaluate our approach to assess XLT and G-XLT capabilities in extractive QA. Results reveal that our self-knowledge distillation approach outperforms standard cross-entropy fine-tuning by a significant margin. Importantly, when compared to a strong baseline that leverages a sizeable volume of machine-translated data, our approach shows competitive results despite the considerable challenge of operating within resource-constrained settings, even in zero-shot scenarios. Beyond performance improvements, we offer valuable insights through comprehensive analyses and an ablation study, further substantiating the benefits and constraints of our approach. In essence, we propose a practical solution to improve cross-lingual QA transfer by leveraging a few data resources in an efficient way.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
