arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.15012cs.CLcs.AIcs.LG
收藏

冷启动个性化通过从结构化世界模型中训练无关先验进行个性化

Cold-Start Personalization via Bayesian Adaptive Questioning

Meta;University of Washington(华盛顿大学);Allen Institute for AI(人工智能研究院)

Avinandan Bose, Shuyue Stella Li, Faeze Brahman, Pang Wei Koh, Simon Shaolei Du, Yulia Tsvetkov, Maryam Fazel, Lin Xiao, Asli Celikyilmaz

AI总结:

Pep通过结构化世界模型和贝叶斯推断实现冷启动个性化,相比强化学习更高效且能更准确预测用户偏好。

摘要:

冷启动个性化需要在没有用户特定历史数据的情况下通过交互推断用户偏好。核心挑战是一个路由问题:每个任务有数十个偏好维度,但个体用户只关心少数几个,而哪些重要取决于谁在提问。在有限的问题预算下,没有结构的提问会错过重要的维度。强化学习是自然的公式化,但在多轮设置中,其终端奖励未能利用偏好数据的因式分解、每标准结构,实际学习的策略会崩溃为静态的问题序列,忽略了用户响应。我们提出将冷启动引导分解为离线结构学习和在线贝叶斯推断。Pep(带有先验的偏好引导)从完整档案中离线学习偏好相关性的结构化世界模型,然后进行无训练的贝叶斯推断,以选择信息性的问题并预测完整的偏好档案,包括从未询问过的维度。该框架在下游求解器中是模块化的,并且只需要简单的信念模型。在医学、数学、社会和常识推理中,Pep在生成响应与用户声明的偏好之间实现了80.8%的一致性,而强化学习为68.5%,交互次数少3-5倍。当两个用户对同一问题给出不同答案时,Pep有39-62%的时间会改变其后续问题,而强化学习为0-28%。它仅需约10K参数,而强化学习为8B,表明冷启动引导的瓶颈是利用偏好数据的因式分解结构的能力。

英文摘要:

Cold-start personalization requires inferring preferences from minimal interaction when no user-specific historical data is available. The space of possible preferences is vast, yet users care about only a sparse subset and rarely articulate them upfront; combined with limited interaction budgets, this makes preference elicitation challenging. Our key insight is that preferences exhibit predictable structure across populations; e.g., users who want detailed explanations often also value worked examples. We propose CAPE (Cold-start Adaptive Preference Elicitation), which learns a structured world model of preference correlations offline from complete profiles, then performs training-free Bayesian inference online to select informative questions and predict complete preference profiles, including dimensions never asked about. Even simple belief model instantiations (e.g., linear regression) substantially outperform end-to-end RL. Across medical, mathematical, social, and commonsense reasoning, CAPE achieves 80.8% alignment with ground-truth user preferences versus 68.5% for RL, requires 2.5x to over 15x fewer interactions, and adapts its questions twice as often as RL. Our contribution is a principled decomposition of cold-start personalization that makes Bayesian preference elicitation practical at scale for LLM systems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑