SemBridge: 通过多语言语义桥实现稀疏编码器中的语言迁移
SemBridge: Language Transfer in Sparse Encoders via Multilingual Semantic Bridges
AI总结:
提出SemBridge方法,利用多语言稠密嵌入作为语义桥,通过选择源语言核心同义词线性组合初始化目标语言词项,解决稀疏编码器在非英语语言上的迁移难题,实验表明在零样本和微调后检索性能均优于基线。
摘要:
稀疏编码器通过在词汇空间内表示词项重要性来实现高精度检索,但其以英语为中心的结构对非英语语言的语言迁移构成了关键障碍。为了克服这一结构限制,我们提出了SemBridge,一种新颖的嵌入初始化方法,通过利用多语言桥模型实现稀疏编码器的跨语言适应。SemBridge使用多语言稠密嵌入作为桥梁,建立源词汇和目标词汇之间的语义对齐。SemBridge不直接依赖所有源词项,而是选择一小部分语义相关的源语言词项,并用它们初始化每个目标语言词项,有效过滤语义噪声,并将目标词项重构为核心同义词的精确线性组合。这加速了微调过程中的收敛,并提高了训练效率。跨五种语言和四种稀疏架构的大量实验表明,与现有基线相比,SemBridge实现了卓越的零样本检索性能,并在微调后持续提升检索性能。这些结果验证了SemBridge作为在不同语言环境中部署高性能稀疏检索系统的实用解决方案。
英文摘要:
Sparse encoders offer high-precision retrieval by representing term importance within a vocabulary space, yet their English-centric structures pose a critical impediment to language transfer for non-English languages. To overcome this structural limitation, we propose SemBridge, a novel embedding initialization method designed for cross-lingual adaptation in sparse encoders by leveraging multilingual bridge models. SemBridge establishes semantic alignments between source and target vocabularies using multilingual dense embeddings as a bridge. Rather than directly relying on all source tokens, SemBridge selects a small set of semantically related source-language tokens and uses them to initialize each target-language token, effectively filtering out semantic noise and reconstructing target tokens as precise linear combinations of core synonyms. This accelerates convergence during fine-tuning and improves training efficiency. Extensive experiments across five languages and four sparse architectures demonstrate that SemBridge achieves superior zero-shot retrieval performance and consistently improves retrieval performance after fine-tuning compared to existing baselines. These results validate SemBridge as a practical solution for deploying high-performance sparse retrieval systems in diverse linguistic environments.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。