arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2601.09515cs.CL
收藏

SERM:具有代理驱动学习的自演化相关性模型

SERM: Self-Evolving Relevance Model with Agent-Driven Learning from Massive Query Streams

Northeastern University, Shenyang, China(东北大学,沈阳,中国);ByteDance(字节跳动);Peking University(北京大学)

Chenglong Wang, Canjia Li, Xingzhao Zhu, Yifu Huo, Huiyu Wang, Weixiong Lin, Yun Yang, Qiaozhi He, Tianhua Zhou, Xiaojia Chang, Jingbo Zhu, Tong Xiao

AI总结:

针对大规模查询流中相关性模型泛化能力不足的问题,SERM通过多代理模块检测分布偏移并生成可靠标签,实现迭代自演化提升性能。

摘要:

由于现实世界查询流的动态演变特性,相关性模型难以泛化到实际搜索场景。一种复杂解决方案是自演化技术。然而在大规模工业环境中,这一技术面临两个挑战:(1)信息样本往往稀疏且难以识别;(2)当前模型生成的伪标签可能不可靠。为了解决这些挑战,本文提出了一种自演化相关性模型方法(SERM),包含两个互补的多代理模块:多代理样本挖掘器,用于检测分布偏移并识别信息训练样本;多代理相关性标注器,通过两级同意框架提供可靠标签。我们在大规模工业环境中评估了SERM,该环境每日处理数十亿用户请求。实验结果表明,SERM通过迭代自演化实现了显著的性能提升,经广泛离线多语言评估和在线测试验证。

英文摘要:

Due to the dynamically evolving nature of real-world query streams, relevance models struggle to generalize to practical search scenarios. A sophisticated solution is self-evolution techniques. However, in large-scale industrial settings with massive query streams, this technique faces two challenges: (1) informative samples are often sparse and difficult to identify, and (2) pseudo-labels generated by the current model could be unreliable. To address these challenges, in this work, we propose a Self-Evolving Relevance Model approach (SERM), which comprises two complementary multi-agent modules: a multi-agent sample miner, designed to detect distributional shifts and identify informative training samples, and a multi-agent relevance annotator, which provides reliable labels through a two-level agreement framework. We evaluate SERM in a large-scale industrial setting, which serves billions of user requests daily. Experimental results demonstrate that SERM can achieve significant performance gains through iterative self-evolution, as validated by extensive offline multilingual evaluations and online testing.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑