RAD-DPO：面向电商生成检索的鲁棒自适应去噪直接偏好优化
RAD-DPO: Robust Adaptive Denoising Direct Preference Optimization for Generative Retrieval in E-commerce
AI总结：
针对生成检索中直接偏好优化的局限性，提出RAD-DPO方法，通过梯度分离保护前缀结构、动态奖励加权缓解标签噪声、全局对比学习扩展正样本覆盖，提升检索精度和训练效率。
摘要：
生成检索（GR）通过结构化语义ID（SIDs）的自回归解码，正在迅速改变电商搜索。尽管架构效率高，但将GR模型与细微的现实用户偏好对齐仍是关键挑战。虽然直接偏好优化（DPO）提供了高效的对齐解决方案，但直接应用于结构化SIDs存在三个限制：（i）惩罚共享的层次前缀，导致梯度冲突；（ii）易受隐含反馈的噪声伪负样本影响；（iii）在多标签查询中有多个相关项目时，会加剧有效候选者之间的概率'挤压效应'。为解决这些问题，我们提出了RAD-DPO，引入了token级梯度分离以保护前缀结构，基于相似性的动态奖励加权以缓解标签噪声，并整合全局SFT损失的多标签全局对比学习目标以显式扩展正样本覆盖。广泛的离线评估和京东核心搜索引擎的大规模在线A/B测试表明，RAD-DPO在检索精度和训练效率上均取得显著提升，证明了其在大规模工业部署中的鲁棒性。
英文摘要：
Generative Retrieval (GR) is rapidly transforming e-commerce search by replacing traditional multi-stage pipelines with the autoregressive decoding of structured Semantic IDs (SIDs). Despite this architectural efficiency, aligning GR models with nuanced, real-world user preferences remains a critical challenge. While Direct Preference Optimization (DPO) offers an efficient alignment solution, its direct application to structured SIDs suffers from three limitations: (i) it penalizes shared hierarchical prefixes, causing gradient conflicts; (ii) it is vulnerable to noisy pseudo-negatives from implicit feedback; and (iii) in multi-label queries with multiple relevant items, it exacerbates a probability "squeezing effect" among valid candidates. To address these issues, we propose RAD-DPO, which introduces token-level gradient detachment to protect prefix structures, similarity-based dynamic reward weighting to mitigate label noise, and a multi-label global contrastive objective integrated with global SFT loss to explicitly expand positive coverage. Extensive offline evaluations and large-scale online A/B testing on JD.com's core search engine demonstrate that RAD-DPO achieves significant improvements in both retrieval precision and training efficiency, proving its robustness for massive industrial deployments.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
