arXiv 2603.104732026-03-11cs.CLcs.AI

将大型语言模型对齐于搜索者偏好

Aligning Large Language Models with Searcher Preferences

School of Artificial Intelligence and Data Science, University of Science and Technology of China（中国科学技术大学人工智能与数据科学学院）；Xiaohongshu Inc.（小红书公司）；Thrust of Artificial Intelligence, The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州）人工智能研究所）；Department of Computer Science and Engineering, The Hong Kong University of Science and Technology（香港科技大学计算机科学与工程系）

Wei Wu, Peilun Zhou, Liyi Chen, Qimeng Wang, Chengqiang Lu, Yan Gao, Yi Wu, Yao Hu, Hui Xiong

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

SearchLLM通过分层多维奖励系统提升开放式生成搜索的鲁棒性和用户需求对齐能力，实测有效消费率提升1.03%。

摘要：

从以项目为中心的排序到以答案为中心的合成的范式转变正在重新定义搜索引擎的作用。尽管最近的工业进展已将生成技术应用于电子商务中的闭合集项目排序，但研究和部署大型内容平台上开放式生成搜索仍受到限制。这种设置引入了挑战，包括对噪声检索的鲁棒性、不可协商的安全保证以及与多样化用户需求的对齐。在本工作中，我们介绍了SearchLLM，这是首个用于开放式生成搜索的大型语言模型（LLM）。我们设计了一个分层、多维奖励系统，将底线约束（包括事实基础、基本答案质量和格式合规性）与促进对噪声检索的鲁棒性和与用户需求对齐的行为优化目标分开。具体而言，我们的奖励模型评估响应，条件是用户查询、会话历史和检索证据集，结合基于规则的检查与人类校准的LLM法官，产生一个可解释的分数向量。我们引入了门控聚合策略，以推导用于优化SearchLLM的训练奖励，使用分组相对策略优化（GRPO）。我们将在RedNote的AI搜索入口部署SearchLLM。离线评估和在线A/B测试显示生成质量提高，用户参与度提高，有效消费率提高1.03%，重新搜索率降低2.81%，同时保持严格的安全部署标准。

英文摘要：

The paradigm shift from item-centric ranking to answer-centric synthesis is redefining the role of search engines. While recent industrial progress has applied generative techniques to closed-set item ranking in e-commerce, research and deployment of open-ended generative search on large content platforms remain limited. This setting introduces challenges, including robustness to noisy retrieval, non-negotiable safety guarantees, and alignment with diverse user needs. In this work, we introduce SearchLLM, the first large language model (LLM) for open-ended generative search. We design a hierarchical, multi-dimensional reward system that separates bottom-line constraints, including factual grounding, basic answer quality and format compliance, from behavior optimization objectives that promote robustness to noisy retrieval and alignment with user needs. Concretely, our reward model evaluates responses conditioned on the user query, session history, and retrieved evidence set, combining rule-based checks with human-calibrated LLM judges to produce an interpretable score vector over these dimensions. We introduce a Gated Aggregation Strategy to derive the training reward for optimizing SearchLLM with Group Relative Policy Optimization (GRPO). We deploy SearchLLM in the AI search entry of RedNote. Offline evaluations and online A/B tests show improved generation quality and user engagement, increasing Valid Consumption Rate by 1.03% and reducing Re-search Rate by 2.81%, while upholding strict safety and reliability standards.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。