PersonaVLM：长期个性化多模态大语言模型
PersonaVLM: Long-Term Personalized Multimodal LLMs
AI总结：
本文提出PersonaVLM，一种支持长期个性化多模态大语言模型框架，通过记忆、推理和响应对齐三大能力，提升个性化交互效果，并在Persona-MME基准测试中取得显著提升。
摘要：
多模态大语言模型（MLLMs）已成为数百万用户的日常助手，但其生成响应与个体偏好一致的能力仍有限。以往方法仅能通过输入增强或输出对齐实现静态、单轮个性化，无法捕捉用户随时间变化的偏好和性格（见图1）。本文提出PersonaVLM，一种创新的个性化多模态代理框架，旨在实现长期个性化。它通过整合三项关键能力将通用多模态大语言模型转化为个性化助手：（a）记忆：主动提取并总结交互中的时间序列多模态记忆，将其整合为个性化数据库；（b）推理：通过检索和整合数据库中的相关记忆进行多轮推理；（c）响应对齐：推断用户在长期交互中的演变性格，确保输出与其独特特征保持一致。为评估，我们建立了Persona-MME，一个包含超过2000个精心挑选交互案例的综合基准，用于评估多模态大语言模型在七个关键方面和14个细粒度任务上的长期个性化能力。广泛实验验证了我们方法的有效性，在128k上下文中，相比基线提升22.4%（Persona-MME）和9.8%（PERSONAMEM），同时在GPT-4o上分别高出5.2%和2.0%。项目页面：https://PersonaVLM.github.io。
英文摘要：
Multimodal Large Language Models (MLLMs) serve as daily assistants for millions. However, their ability to generate responses aligned with individual preferences remains limited. Prior approaches enable only static, single-turn personalization through input augmentation or output alignment, and thus fail to capture users' evolving preferences and personality over time (see Fig.1). In this paper, we introduce PersonaVLM, an innovative personalized multimodal agent framework designed for long-term personalization. It transforms a general-purpose MLLM into a personalized assistant by integrating three key capabilities: (a) Remembering: It proactively extracts and summarizes chronological multimodal memories from interactions, consolidating them into a personalized database. (b) Reasoning: It conducts multi-turn reasoning by retrieving and integrating relevant memories from the database. (c) Response Alignment: It infers the user's evolving personality throughout long-term interactions to ensure outputs remain aligned with their unique characteristics. For evaluation, we establish Persona-MME, a comprehensive benchmark comprising over 2,000 curated interaction cases, designed to assess long-term MLLM personalization across seven key aspects and 14 fine-grained tasks. Extensive experiments validate our method's effectiveness, improving the baseline by 22.4% (Persona-MME) and 9.8% (PERSONAMEM) under a 128k context, while outperforming GPT-4o by 5.2% and 2.0%, respectively. Project page: https://PersonaVLM.github.io.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
