MMPB：是时候推进多模态个性化了
MMPB: It's Time for Multi-Modal Personalization
AI总结：
该研究提出首个大规模VLM个性化基准MMPB，通过111个概念、1万对图像-查询和三阶段协议评估23个模型，揭示其在一致性、偏好处理和视觉适应上的不足。
摘要：
视觉个性化在面向用户的AI系统（如智能家居和医疗保健）中至关重要，在这些场景中，使模型行为与以用户为中心的概念保持一致非常关键。然而，近期的大型视觉-语言模型（VLM）尽管具有广泛适用性，但其适应单个用户的能力仍未得到充分研究。本文提出MMPB，这是首个用于评估VLM个性化能力的大规模基准。MMPB包含1万个图像-查询对，涵盖人类、动物、物体和角色四大类别中的111个可个性化概念，其中人类类别还加入了基于偏好的查询。我们将个性化组织为三种主要任务类型，每种任务突出VLM的一项不同关键属性。我们使用23种广泛采用的开放源代码和闭源代码VLM，通过概念注入、多轮对话和个性化查询这一三阶段协议评估个性化性能。研究结果表明，大多数VLM（包括部分闭源模型）在个性化方面表现困难，尤其是在对话过程中保持一致性、处理用户偏好以及适应视觉线索方面。我们的分析揭示，VLM个性化中的挑战（如拒绝行为和长上下文遗忘）表明其仍有相当大的改进空间。通过识别这些局限并提供可扩展的基准，MMPB为未来迈向真正个性化的多模态AI研究提供了宝贵洞见和坚实基础。项目页面：aidaslab.github.io/MMPB
英文摘要：
Visual personalization is essential in user-facing AI systems such as smart homes and healthcare, where aligning model behavior with user-centric concepts is critical. However, recent large Vision-Language Models (VLMs), despite their broad applicability, remain underexplored in their ability to adapt to individual users. In this paper, we introduce MMPB, the first extensive benchmark for evaluating VLMs on personalization. MMPB comprises 10k image-query pairs and includes 111 personalizable concepts across four categories: humans, animals, objects, and characters, with the human category enriched with preference-grounded queries. We structure personalization into three main task types, each highlighting a different key property of VLMs. Using 23 widely used VLMs including both open- and closed-source models, we evaluate personalization performance via a three-stage protocol: concept injection, multi-turn dialogue, and personalized querying. Our findings indicate that most VLMs (including some closed-source models) struggle with personalization, particularly in maintaining consistency over dialogue, handling user preferences, and adapting to visual cues. Our analysis reveals that the challenges in VLM personalization (such as refusal behaviors and long-context forgetting) highlight substantial room for improvement. By identifying these limitations and offering a scalable benchmark, MMPB offers valuable insights and a solid foundation for future research toward truly personalized multi-modal AI. Project Page: aidaslab.github.io/MMPB
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
