Vision-Language Models (VLMs) are increasingly utilized as the conditioning backbone for diffusion-based image editing due to their remarkable multimodal reasoning capabilities. While standalone VLMs demonstrate strong localization capabilities, editing pipelines frequently struggle to maintain this accuracy, particularly in complex, multi-entity scenes. In this work, we investigate this performance gap, hypothesizing that it stems from treating the VLM as a condition encoder. In this role, the model is restricted to a single forward pass, preventing the autoregressive generation process for which it was optimized, thereby failing to fully expose its capabilities. To investigate whether this spatial understanding persists when the VLM is used as a condition encoder, we introduce Analysis-by-Proxy. In this framework, we train a lightweight, interpretable proxy model on the VLM's intermediate representations using an auxiliary localization task. By analyzing the VLM through this proxy, we uncover the specific VLM representations that encode localization information. Our findings expose a fundamental mismatch between how spatial knowledge is represented within a VLM condition encoder and how it is extracted by current editing pipelines. We reveal that under single-pass constraints, the localization signal does not reliably propagate to the predefined layer configurations commonly used for conditioning. Instead, this crucial signal remains hidden within intermediate representations, at locations that vary depending on the input prompt. Using our introduced Analysis-by-Proxy framework, we reveal the fundamental failures of existing condition extraction strategies in editing pipelines, opening the door to more principled design of conditioning architectures.
Reading Smiles: Proxy Bias in Foundation Models for Facial Emotion Recognition
CHI – Chair of Health Informatics, MRI, Technical University of Munich(慕尼黑技术大学健康信息学系);MCML – Munich Center for Machine Learning(慕尼黑机器学习中心);GLAM – Group on Language, Audio, & Music(语言、音频与音乐小组);MDSI – Munich Data Science Institute(慕尼黑数据科学研究所)
Gated Relational Alignment via Confidence-based Distillation for Efficient VLMs
Department of Information Technology(信息科技系);Electrical Engineering, ETH Zurich, Zurich, Switzerland(电气工程,苏黎世联邦理工学院,苏黎世,瑞士);Qualcomm AI Research, Amsterdam, the Netherlands(高通人工智能研究,阿姆斯特丹,荷兰);Department of Electrical, Electronic and Information Engineering(电气、电子与信息工程系);University of Bologna, Bologna, Italy(博洛尼亚大学,博洛尼亚,意大利);School of Electrical and Electronic Engineering(电气与电子工程学院)
Operating Within the Operational Design Domain: Zero-Shot Perception with Vision-Language Models
Volkswagen Aktiengesellschaft(大众汽车股份有限公司);L3S Research Center(莱比锡大学汉诺威研究中心);Faculty of Information Technology(信息科技学院);MOIA GmbH(MOIA公司);Motor AI GmbH(Motor AI公司)
Gold Points Sniper: Self-guided Visual Reasoning in VLM for Fine-grained Action Understanding
Beijing National Research Center for Information Science and Technology (BNRist), Department of Automation, Tsinghua University(清华大学自动化系北京信息科学与技术国家研究中心);State Key Laboratory of General Artificial Intelligence, Beijing Institute for General Artificial Intelligence (BIGAI)(北京通用人工智能研究院通用人工智能国家重点实验室)
Thinking Ahead: Foresight Intelligence in MLLMs and World Model
College of Software, Nankai University, China(南开大学软件学院);The University of Hong Kong, China(香港大学);NKIARI, Shenzhen Futian, China(NKIARI,深圳福田,中国);AAIS & VCIP, Nankai University, China(AAIS与VCIP,南开大学,中国);A*STAR Institute of Advanced Intelligence and Computing, Singapore(新加坡A*STAR先进智能与计算研究所)
Look-Closer-Then-Diagnose: Confidence-Aware Ultrasound VQA via Active Zooming
Computer Aided Medical Procedures (CAMP)(计算机辅助医疗程序);TU Munich, Germany(慕尼黑工业大学,德国);Munich Center for Machine Learning (MCML)(慕尼黑机器学习中心);Munich, Germany(慕尼黑,德国);Zhongshan Hospital, Fudan University, China(复旦大学中山医院);The University of Hong Kong, Hongkong, China(香港大学,香港,中国)
Boosting Vision-Language Models for Histopathology Classification: Predict all at once
Université Catholique de Louvain (UCLouvain)(天主教鲁汶大学(UCLouvain));Université de Mons (UMons)(蒙斯大学(UMons));École de Technologie Supérieure (ÉTS)(高等技术学院(ÉTS));Centre de Recherche du Centre Hospitalier de l’Université de Montréal (CRCHUM)(蒙特利尔大学中心医院研究中心(CRCHUM))
Translating Inference-Time Control to Radiology Vision-Language Models: Activation Steering for Pneumonia Classification on Chest X-rays
Universidade Federal de São Paulo (UNIFESP)(圣保罗联邦大学);Hospital Israelita Albert Einstein(以色列阿尔伯特·爱因斯坦医院);Stanford University School of Medicine(斯坦福大学医学院);DASA;University of Texas Southwestern Medical Center (UTSW)(德克萨斯大学西南医学中心);Eden
Visual serial processing deficits explain divergences in human and VLM reasoning
Princeton Neuroscience Institute(普林斯顿神经科学研究所);Department of Psychology, Princeton University(普林斯顿大学心理学系);Department of Psychology, Université de Montréal(蒙特利尔大学心理学系);Mila - Quebec AI Institute(魁北克AI研究所);Department of Computer Science, Princeton University(普林斯顿大学计算机科学系)
ComMem: Complementary Memory Systems for Test-Time Adaptation of Vision-Language Models
School of Life Sciences, IDG/McGovern Institute for Brain Research, Tsinghua University, Beijing, China(生命科学学院,IDG/麦克戈文脑科学研究院,清华大学,北京,中国);Institute of Software Chinese Academy of Sciences, Beijing, China(中国科学院软件研究所,北京,中国);Beijing Academy of Artificial Intelligence, Beijing, China(北京人工智能研究院,北京,中国);Department of Psychological and Cognitive Sciences, Tsinghua University, Beijing, China(心理学与认知科学系,清华大学,北京,中国);Dept. of Comp. Sci. and Tech., Institute for AI, Tsinghua-Bosch Joint ML Center, THBI Lab, BNRist Center, Tsinghua University, Beijing, China(计算机科学与技术系,人工智能研究院,清华大学-博世联合机器学习中心,THBI实验室,BNRist中心,清华大学,北京,中国)
From Segments to Scenes: Temporal Understanding for Agentic Autonomous Driving via Vision-Language Models
University of California, Berkeley(加州大学伯克利分校);University of Cambridge(剑桥大学);University of Toronto(多伦多大学);ETH Zurich(苏黎世联邦理工学院);University of Washington(华盛顿大学);University of Southern California(南加州大学)
MedFoundationHub: A Lightweight and Secure Toolkit for Deploying Medical Vision Language Foundation Models
Vanderbilt University(范德比尔特大学);Weill Cornell Medicine(韦尔·科恩医学中心);Vanderbilt University Medical Center(范德比尔特大学医学中心);UT MD Anderson Cancer Center(德克萨斯大学MD安德森癌症中心)