通知：联邦节假日调休：哥伦布日 / 原住民日（Columbus Day / Indigenous Peoples' Day），今日（13日）无更新。通知：联邦节假日调休：哥伦布日 / 原住民日（Columbus Day / Indigenous Peoples' Day），今日（13日）无更新。
arXiv 2602.03454cs.CV
基于上下文的视觉个性化在视觉-语言模型中
Contextualized Visual Personalization in Vision-Language Models
Department of Electrical and Computer Engineering, Seoul National University, Seoul, South Korea（电气电子工程系，首尔国立大学，首尔，韩国）；Interdisciplinary Program in Artificial Intelligence, Seoul National University, Seoul, Korea（人工智能交叉学科项目，首尔国立大学，首尔，韩国）
Yeongtak Oh, Sangwon Yu, Junsung Park, Han Cheol Moon, Jisoo Mok, Sungroh Yoon
Despite recent progress in vision-language models (VLMs), existing approaches often fail to generate personalized responses based on the user's specific experiences, as they lack the ability to associate visual inputs with a user's accumulated visual-textual context. We newly formalize this challenge as contextualized visual personalization, which requires the visual recognition and textual retrieval of personalized visual experiences by VLMs when interpreting new images. To address this issue, we propose CoViP, a unified framework that treats personalized image captioning as a core task for contextualized visual personalization and improves this capability through reinforcement-learning-based post-training and caption-augmented generation. We further introduce diagnostic evaluations that explicitly rule out textual shortcut solutions and verify whether VLMs truly leverage visual context. Extensive experiments demonstrate that existing open-source and proprietary VLMs exhibit substantial limitations, while CoViP not only improves personalized image captioning but also yields holistic gains across downstream personalization tasks. These results highlight CoViP as a crucial stage for enabling robust and generalizable contextualized visual personalization.
CoViPAL: Layer-wise Contextualized Visual Token Pruning for Large Vision-Language Models
School of Artificial Intelligence, Wuhan University（武汉大学人工智能学院）；School of Computer Science, Wuhan University（武汉大学计算机学院）；School of Computer Science, Shanghai Jiao Tong University（上海交通大学计算机学院）；Cognitive AI Lab, Shanghai Huawei Technologies, China（上海华为技术有限公司认知人工智能实验室）
Mod-Adapter: Tuning-Free and Versatile Multi-concept Personalization via Modulation Adapter
School of Computer Science and Engineering, Sun Yat-sen University（中山大学计算机科学与工程学院）；Kolors Team, Kuaishou Technology（快手科技Kolors团队）；Shenzhen Loop Area Institute, Shenzhen, China（深圳循环区研究所）；Guangdong Key Laboratory of Big Data Analysis and Processing, Guangzhou, China（广东大数据分析与处理重点实验室）；South China University of Technology, Guangzhou, China（华南理工大学）
University of British Columbia（不列颠哥伦比亚大学）；Vector Institute for AI（向量人工智能研究所）
There has been a growing interest in solving Visual Question Answering (VQA) tasks that require the model to reason beyond the content present in the image. In this work, we focus on questions that require commonsense reasoning. In contrast to previous methods which inject knowledge from static knowledge bases, we investigate the incorporation of contextualized knowledge using Commonsense Transformer (COMET), an existing knowledge model trained on human-curated knowledge bases. We propose a method to generate, select, and encode external commonsense knowledge alongside visual and textual cues …
Omni-Persona: Systematic Benchmarking and Improving Omnimodal Personalization
Department of Electrical and Computer Engineering, Seoul National University（首尔国立大学电气与计算机工程系）；Interdisciplinary Program in Artificial Intelligence, Seoul National University（首尔国立大学人工智能跨学科项目）；Department of Artificial Intelligence, University of Seoul（首尔大学人工智能系）
One-to-Normal: Anomaly Personalization for Few-shot Anomaly Detection
West China Hospital, Sichuan University（四川大学华西医院）；School of Artificial Intelligence, Beijing University of Posts and Telecommunications（北京邮电大学人工智能学院）；Sichuan University Pittsburgh Institute, Sichuan University（四川大学匹兹堡学院）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）
CLIP-AUTT: Test-Time Personalization with Action Unit Prompting for Fine-Grained Video Emotion Recognition
LIVIA, ILLS, Dept. of Systems Engineering, ETS Montreal（LIVIA, ILLS, 系统工程学院, 蒙特利尔高等技术学院）；LIVIA, Dept. of Software and IT Engineering, ETS Montreal（LIVIA, 软件与IT工程学院, 蒙特利尔高等技术学院）；Dept. of Computer Science, École Polytechnique（计算机科学系, 巴黎综合理工学院）
Carnegie Mellon University（卡内基梅隆大学）；Facebook AI；University of Oxford（牛津大学）
This paper studies zero-shot cross-lingual transfer of vision-language models. Specifically, we focus on multilingual text-to-video search and propose a Transformer-based model that learns contextualized multilingual multimodal embeddings. Under a zero-shot setting, we empirically demonstrate that performance degrades significantly when we query the multilingual text-video model with non-English sentences. To address this problem, we introduce a multilingual multimodal pre-training strategy, and collect a new multilingual instructional video dataset (MultiHowTo100M) for pre-training. Experimen…
School of Information, University of Arizona（亚利桑那大学信息学院）
The recently proposed ALFRED challenge task aims for a virtual robotic agent to complete complex multi-step everyday tasks in a virtual home environment from high-level natural language directives, such as "put a hot piece of bread on a plate". Currently, the best-performing models are able to complete less than 5% of these tasks successfully. In this work we focus on modeling the translation problem of converting natural language directives into detailed multi-step sequences of actions that accomplish those goals in the virtual environment. We empirically demonstrate that it is possible to ge…
CLUTCH: Contextualized Language model for Unlocking Text-Conditioned Hand motion modelling in the wild
Technical University of Darmstadt（德累斯顿技术大学）；Max-Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；University of Tuebingen（图宾根大学）；Tubingen AI Center（图宾根人工智能中心）；Max Planck Institute for Informatics（马克斯·普朗克信息研究所）
VisMem: Latent Vision Memory Unlocks Potential of Vision-Language Models
National University of Singapore（新加坡国立大学）；Fudan University（复旦大学）；Tsinghua University（清华大学）；Zhejiang University（浙江大学）；University of Science and Technology of China（中国科学技术大学）；vivo
In this work, we propose an open-vocabulary object detection method that, based on image-caption pairs, learns to detect novel object classes along with a given set of known classes. It is a two-stage training approach that first uses a location-guided image-caption matching technique to learn class labels for both novel and known classes in a weakly-supervised manner and second specializes the model for the object detection task using known class annotations. We show that a simple language model fits better than a large contextualized language model for detecting novel objects. Moreover, we i…
