多语言生成式检索:基于跨语言语义压缩
Multilingual Generative Retrieval via Cross-lingual Semantic Compression
AI总结:
提出MGR-CSC框架,通过跨语言语义压缩统一多语言关键词为共享原子,并采用动态多步约束解码,解决多语言生成式检索中的标识符错位与膨胀问题,显著提升检索精度并压缩标识符长度。
摘要:
生成式信息检索是一种新兴的检索范式,在单语场景中表现出卓越的性能。然而,将这些方法应用于多语言检索仍面临两大主要挑战:跨语言标识符错位和标识符膨胀。为解决这些局限,我们提出了基于跨语言语义压缩的多语言生成式检索(MGR-CSC),这是一种新颖的框架,它将语义等价的多语言关键词统一为共享原子以对齐语义并压缩标识符空间,同时我们提出了一种检索过程中的动态多步约束解码策略。MGR-CSC通过分配一致的标识符改善了跨语言对齐,并通过减少冗余提高了解码效率。实验表明,MGR-CSC取得了出色的检索准确率,在mMarco100k上提升了6.83%,在mNQ320k上提升了4.77%,同时分别将文档标识符长度减少了74.51%和78.2%。
英文摘要:
Generative Information Retrieval is an emerging retrieval paradigm that exhibits remarkable performance in monolingual scenarios.However, applying these methods to multilingual retrieval still encounters two primary challenges, cross-lingual identifier misalignment and identifier inflation. To address these limitations, we propose Multilingual Generative Retrieval via Cross-lingual Semantic Compression (MGR-CSC), a novel framework that unifies semantically equivalent multilingual keywords into shared atoms to align semantics and compresses the identifier space, and we propose a dynamic multi-step constrained decoding strategy during retrieval. MGR-CSC improves cross-lingual alignment by assigning consistent identifiers and enhances decoding efficiency by reducing redundancy. Experiments demonstrate that MGR-CSC achieves outstanding retrieval accuracy, improving by 6.83% on mMarco100k and 4.77% on mNQ320k, while reducing document identifiers length by 74.51% and 78.2%, respectively.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。