从人类反馈中学习个性化代理
Learning Personalized Agents from Human Feedback
AI总结：
PAHF通过整合显式记忆与双反馈通道，实现持续个性化，显著提升学习速度和适应能力。
摘要：
现代AI代理功能强大，但往往无法与个体用户的独特、不断变化的偏好对齐。先前的方法通常依赖静态数据集，要么在交互历史上训练隐式偏好模型，要么在外部内存中编码用户资料。然而，这些方法在新用户和随时间变化的偏好方面存在困难。我们引入了从人类反馈中学习个性化代理（PAHF）框架，该框架用于持续个性化，其中代理通过实时交互学习使用显式个人记忆。PAHF操作化了一个三步循环：（1）寻求预行动澄清以解决歧义，（2）将行动基于从记忆中检索的偏好进行定位，（3）整合后行动反馈以更新记忆当偏好漂移时。为了评估这种能力，我们开发了一个四阶段协议和两个在具身操作和在线购物中的基准测试。这些基准测试量化了代理学习初始偏好从头开始以及随后适应身份转变的能力。我们的理论分析和实证结果表明，整合显式记忆与双反馈通道是关键：PAHF学习速度显著加快，并且在无记忆和单通道基线中表现一致，减少了初始个性化误差，并能够快速适应偏好变化。
英文摘要：
Modern AI agents are powerful but often fail to align with the idiosyncratic, evolving preferences of individual users. Prior approaches typically rely on static datasets, either training implicit preference models on interaction history or encoding user profiles in external memory. However, these approaches struggle with new users and with preferences that change over time. We introduce Personalized Agents from Human Feedback (PAHF), a framework for continual personalization in which agents learn online from live interaction using explicit per-user memory. PAHF operationalizes a three-step loop: (1) seeking pre-action clarification to resolve ambiguity, (2) grounding actions in preferences retrieved from memory, and (3) integrating post-action feedback to update memory when preferences drift. To evaluate this capability, we develop a four-phase protocol and two benchmarks in embodied manipulation and online shopping. These benchmarks quantify an agent's ability to learn initial preferences from scratch and subsequently adapt to persona shifts. Our theoretical analysis and empirical results show that integrating explicit memory with dual feedback channels is critical: PAHF learns substantially faster and consistently outperforms both no-memory and single-channel baselines, reducing initial personalization error and enabling rapid adaptation to preference shifts.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
