arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2604.17632cs.IR
收藏

代码切换信息检索:基准测试、分析及当前检索器的局限

Code-Switching Information Retrieval: Benchmarks, Analysis, and the Limits of Current Retrievers

Northwestern University(西北大学);Waseda University(早稻田大学);Brown University(布朗大学);RIKEN AIP(理化学研究所脑科学综合研究中心);Snowflake Inc.(雪佛龙公司);University of Utah(犹他大学);The University of Tokyo(东京大学)

Qingcheng Zeng, Yuheng Lu, Zeqi Zhou, Heli Qi, Puxuan Yu, Fuheng Zhao, Hitomi Yanaka, Weihao Xuan, Naoto Yokoya

AI总结:

研究探讨代码切换在信息检索中的影响,提出CSR-L基准测试,发现代码切换显著降低多语言模型性能,且嵌入空间差异是主因,提出CS-MTEB基准测试显示性能下降达27%。

摘要:

代码切换是全球通信中的普遍现象,但现代信息检索系统仍主要针对单语环境设计。本文提出全面研究代码切换信息检索,引入CSR-L基准测试,通过人工标注构建数据集以捕捉混合语言查询的自然性。评估统计、密集和晚交互范式表明,代码切换是性能瓶颈,降低甚至稳健多语言模型效果。发现嵌入空间差异是根本原因。扩展研究提出CS-MTEB基准测试,涵盖11种任务,观察性能下降最高达27%。标准多语言技术如词汇扩展无法完全解决这些缺陷。研究结果凸显当前系统的脆弱性,确立代码切换为未来信息检索优化的关键领域。

英文摘要:

Code-switching is a pervasive linguistic phenomenon in global communication, yet modern information retrieval systems remain predominantly designed for, and evaluated within, monolingual contexts. To bridge this critical disconnect, we present a holistic study dedicated to code-switching IR. We introduce CSR-L (Code-Switching Retrieval benchmark-Lite), constructing a dataset via human annotation to capture the authentic naturalness of mixed-language queries. Our evaluation across statistical, dense, and late-interaction paradigms reveals that code-switching acts as a fundamental performance bottleneck, degrading the effectiveness of even robust multilingual models. We demonstrate that this failure stems from substantial divergence in the embedding space between pure and code-switched text. Scaling this investigation, we propose CS-MTEB, a comprehensive benchmark covering 11 diverse tasks, where we observe performance declines of up to 27%. Finally, we show that standard multilingual techniques like vocabulary expansion are insufficient to resolve these deficits completely. These findings underscore the fragility of current systems and establish code-switching as a crucial frontier for future IR optimization.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑