Ego：基于嵌入的视觉-语言模型个性化
Ego: Embedding-Guided Personalization of Vision-Language Models
AI总结：
本文提出了一种基于模型内部机制的高效视觉-语言模型个性化方法，通过提取目标概念的视觉标记实现个性化描述与记忆。
摘要：
支持人类日常生活的AI助手正变得越来越可行，这得益于多模态语言模型的快速进步。一个关键挑战在于克服这些模型的通用性，以提供个性化的体验。现有的大规模视觉语言模型个性化方法通常依赖于额外的训练阶段，这限制了通用性和可扩展性，或者依赖于工程化的流水线和外部预训练模块，这阻碍了部署效率。在本工作中，我们提出了一种高效的个性化方法，利用模型自身捕捉个性化概念的能力。具体而言，我们通过利用模型内部的注意力机制提取出主要代表目标概念的视觉标记。这些标记作为该特定概念的记忆，使模型在测试图像中出现时能够回忆并描述它。我们对我们的方法和最先进方法在各种个性化设置中进行了全面且统一的评估，包括单概念、多概念和视频个性化，展示了在最小个性化开销下显著的性能提升。
英文摘要：
AI assistants that support humans in daily life are becoming increasingly feasible, driven by the rapid advancements in multimodal language models. A key challenge lies in overcoming the generic nature of these models to deliver personalized experiences. Existing approaches to personalizing large vision language models often rely on additional training stages, which limit generality and scalability, or on engineered pipelines with external pre-trained modules, which hinder deployment efficiency. In this work, we propose an efficient personalization method that leverages the model's inherent ability to capture personalized concepts. Specifically, we extract visual tokens that predominantly represent the target concept by utilizing the model's internal attention mechanisms. These tokens serve as a memory of that specific concept, enabling the model to recall and describe it when it appears in test images. We conduct a comprehensive and unified evaluation of our approach and SOTA methods across various personalization settings including single-concept, multi-concept, and video personalization, demonstrating strong performance gains with minimal personalization overhead.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
