模块化句子编码器：将语言特化与跨语言对齐分离
Modular Sentence Encoders: Separating Language Specialization from Cross-Lingual Alignment
AI总结：
提出模块化句子编码器，通过先训练单语模块再训练跨语言对齐适配器，分离语言特化与跨语言对齐，解决多语言诅咒和任务冲突，实现更优且均衡的性能。
摘要：
多语言句子编码器（MSEs）通常通过训练多语言语言模型将不同语言的句子映射到共享语义空间来获得。因此，它们受到多语言诅咒的影响，即由于参数共享导致的单语表示准确性损失。MSEs的另一个局限性是不同任务性能之间的权衡：跨语言对齐训练扭曲了各语言语义空间的最佳单语结构，损害了句子嵌入在单语任务中的效用；跨语言任务，如跨语言语义相似性和句子分类的零样本迁移，也可能需要相互冲突的跨语言对齐策略。在这项工作中，我们通过对句子编码器进行模块化训练来解决这两个问题。我们首先训练特定语言的单语模块，以减轻语言之间的负面干扰（即诅咒）。然后，我们通过训练跨语言对齐适配器，将所有非英语句子嵌入与英语对齐，防止对第一步中的单语特化产生干扰。我们使用两种不同类型的数据训练跨语言适配器，以解决不同跨语言任务的冲突要求。在语义文本相似性和相关性、双语文本挖掘和句子分类上的单语和跨语言结果表明，与整体多语言句子编码器的全参数训练相比，我们的模块化解决方案在所有任务中实现了更好且更平衡的性能，尤其有利于低资源语言。
英文摘要：
Multilingual sentence encoders (MSEs) are commonly obtained by training multilingual language models to map sentences from different languages into a shared semantic space. As such, they are subject to curse of multilinguality, a loss of monolingual representational accuracy due to parameter sharing. Another limitation of MSEs is the trade-off between different task performance: cross-lingual alignment training distorts the optimal monolingual structure of semantic spaces of individual languages, harming the utility of sentence embeddings in monolingual tasks; cross-lingual tasks, such as cross-lingual semantic similarity and zero-shot transfer for sentence classification, may also require conflicting cross-lingual alignment strategies. In this work, we address both issues by means of modular training of sentence encoders. We first train language-specific monolingual modules to mitigate negative interference between languages (i.e., the curse). We then align all non-English sentence embeddings to the English by training cross-lingual alignment adapters, preventing interference with monolingual specialization from the first step. We train the cross-lingual adapters with two different types of data to resolve the conflicting requirements of different cross-lingual tasks. Monolingual and cross-lingual results on semantic text similarity and relatedness, bitext mining and sentence classification show that our modular solution achieves better and more balanced performance across all the tasks compared to full-parameter training of monolithic multilingual sentence encoders, especially benefiting low-resource languages.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
