个性化具身多模态大语言模型代理在长期用户交互中的应用
Personalizing Embodied Multimodal Large Language Model Agents over Long-term User Interactions
AI总结：
提出POLAR框架，通过多模态知识图谱记忆机制增强具身代理在长期交互中的个性化能力，显著提升多步推理和用户上下文跟踪性能。
摘要：
基于多模态大语言模型的具身代理在物理环境中解决复杂任务方面展现出强大潜力。然而，个性化辅助不仅需要遵循通用指令或识别物体类别。在现实场景中，目标通常仅通过先前的交互隐式指定，要求代理利用随时间积累的个性化上下文。在这项工作中，我们提出了POLAR，一个用于长期用户交互中个性化具身代理的多模态记忆增强框架。POLAR将先前的交互组织成一个多模态知识图谱，该图谱捕获用于个性化上下文和视觉概念的语义记忆，以及用于代理轨迹等具身经验的 episodic 记忆。为了执行具身任务，POLAR检索相关记忆以解释当前请求并指导任务执行。我们在多个MLLM骨干网络和多样化的评估场景下评估POLAR，以研究记忆在长期个性化中的作用。结果表明，所提出的记忆机制通过更有效地利用先前交互中积累的信息，持续提升性能。当代理需要在多个交互中进行推理、执行多跳推理或随时间跟踪用户特定上下文的更新时，性能提升尤为显著。
英文摘要：
Multimodal large language model (MLLM)-based embodied agents have shown strong potential for solving complex tasks in physical environments. However, personalized assistance requires more than following generic instruction or recognizing object categories. In real-world scenarios, the intended target is often specified only implicitly through prior interactions, requiring agents to leverage personalized context accumulated over time. In this work, we propose POLAR, a multiomodal memory-augmented framework for personalized embodied agents over long-term user interactions. POLAR organizes prior interactions into a multimodal knowledge graph that captures semantic memory for personalized context and visual concepts, and episodic memory for embodied experiences such as agent trajectories. To execute embodied tasks, POLAR retrieves relevant memories to interpret the current request and guide task execution. We evaluate POLAR across multiple MLLM backbones and diverse evaluation scenarios to study the role of memory in long-term personalization. Results show that the proposed memory mechanism consistently improves performance by enabling more effective use of information accumulated over prior interactions. The gains are especially pronounced when the agents are required to reason across multiple interactions, perform multi-hop inference, or tracking updates in user-specific context over time.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
