与1000个角色交谈后:从大规模角色交互中学习偏好对齐的主动助手
After Talking with 1,000 Personas: Learning Preference-Aligned Proactive Assistants From Large-Scale Persona Interactions
AI总结:
通过大规模角色交互模拟,学习偏好对齐的主动助手,实现设备端个性化适应与冷启动优化。
摘要:
智能助手越来越主动地行事,但有时过于及时或侵入性的行为会使用户失去信任并禁用这些功能。学习用户对主动帮助的偏好是困难的,因为现实世界的研究成本高、规模有限,且很少捕捉到多个交互会话中偏好如何变化。基于大型语言模型的生成代理提供了一种模拟真实交互的方法,但现有的合成数据集在时间深度、多样化角色和多维偏好方面仍然有限。它们还很少支持在设备限制下将群体层面的洞察转移到个体用户。我们提出了一个从群体到个体的学习框架,用于偏好对齐的主动助手,该框架在设备和隐私限制下运行。我们的方法利用大规模的交互模拟,使用1000个多样化角色来学习用户在重复维度如时间、自主性和沟通风格中表达偏好的共享结构,从而在不依赖真实用户日志的情况下提供强大的冷启动。助手然后通过轻量级的激活驱动的转向,在设备上通过简单的交互反馈适应个体用户,而无需模型重新训练或云端更新。我们通过受控模拟使用1000个模拟角色和30名参与者的真人研究评估了该框架。结果表明,与未调优和直接响应基线相比,改进了时间决策和感知交互质量,而设备上的激活转向在性能上与基于人类反馈的强化学习相当。参与者还报告了随着交互会话的多次进行,助手适应后更高的满意度、信任和舒适度。
英文摘要:
Smart assistants increasingly act proactively, yet mistimed or intrusive behavior often causes users to lose trust and disable these features. Learning user preferences for proactive assistance is difficult because real-world studies are costly, limited in scale, and rarely capture how preferences change across multiple interaction sessions. Large language model based generative agents offer a way to simulate realistic interactions, but existing synthetic datasets remain limited in temporal depth, diverse personas, and multi-dimensional preferences. They also provide little support for transferring population-level insights to individual users under on-device constraints. We present a population-to-individual learning framework for preference-aligned proactive assistants that operates under on-device and privacy constraints. Our approach uses large-scale interaction simulation with 1,000 diverse personas to learn shared structure in how users express preferences across recurring dimensions such as timing, autonomy, and communication style, providing a strong cold start without relying on real user logs. The assistant then adapts to individual users on device through lightweight activation-based steering driven by simple interaction feedback, without model retraining or cloud-side updates. We evaluate the framework using controlled simulations with 1,000 simulated personas and a human-subject study with 34 participants. Results show improved timing decisions and perceived interaction quality over untuned and direct-response baselines, while on-device activation steering achieves performance comparable to reinforcement learning from human feedback. Participants also report higher satisfaction, trust, and comfort as the assistant adapts over multiple sessions of interactions.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。