使用元蒸馏学习的多语言句子级语义搜索
Multilingual Sentence-Level Semantic Search using Meta-Distillation Learning
AI总结：
本文提出针对低资源场景的MAML-Align方法，利用基于MAML的元蒸馏学习将知识从单语到双语的Teacher模型迁移至双语到多语言的Student模型，显著提升了多语言语义搜索性能及对未见语言的泛化能力。
摘要：
多语言语义搜索是检索与以不同语言组合表达的查询相关内容的任务。这要求对用户的意图及其上下文含义有更好的语义理解。由于缺乏用于此任务的多语言平行资源以及需要规避“语言偏见”，多语言语义搜索比其单语或双语对应物探索得更少且更具挑战性。在这项工作中，我们提出了一种对齐方法：MAML-Align，专门针对低资源场景。我们的方法利用基于MAML的元蒸馏学习，MAML是一种基于优化的与模型无关的元学习器。MAML-Align将知识从专门用于从单语到双语语义搜索转移的Teacher元迁移模型T-MAML，蒸馏到从双语到多语言语义搜索进行元迁移的Student模型S-MAML。据我们所知，我们是第一个将元蒸馏扩展到多语言搜索应用的。我们的实证结果表明，在基于句子变换器的强大基线之上，我们的元蒸馏方法提升了MAML提供的增益，并显著优于朴素的微调方法。此外，多语言元蒸馏学习甚至提高了对未见语言的泛化能力。
英文摘要：
Multilingual semantic search is the task of retrieving relevant contents to a query expressed in different language combinations. This requires a better semantic understanding of the user's intent and its contextual meaning. Multilingual semantic search is less explored and more challenging than its monolingual or bilingual counterparts, due to the lack of multilingual parallel resources for this task and the need to circumvent "language bias". In this work, we propose an alignment approach: MAML-Align, specifically for low-resource scenarios. Our approach leverages meta-distillation learning based on MAML, an optimization-based Model-Agnostic Meta-Learner. MAML-Align distills knowledge from a Teacher meta-transfer model T-MAML, specialized in transferring from monolingual to bilingual semantic search, to a Student model S-MAML, which meta-transfers from bilingual to multilingual semantic search. To the best of our knowledge, we are the first to extend meta-distillation to a multilingual search application. Our empirical results show that on top of a strong baseline based on sentence transformers, our meta-distillation approach boosts the gains provided by MAML and significantly outperforms naive fine-tuning methods. Furthermore, multilingual meta-distillation learning improves generalization even to unseen languages.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
