How well can vision-language models (VLMs) reason about what is missing from an image? We study this question with OccludeBench, a manually curated benchmark for recovering fully occluded objects from the visual evidence that remains. OccludeBench distinguishes direct cases, where the hidden object must be recovered from a specific physical cue such as a shadow or reflection, from inferential cases, where it can be inferred from broader scene context. Across five VLMs, accuracy averages 42.7% on direct cases versus 62.7% on inferential cases, exhibiting a 20.0-point gap. Humans attain higher accuracies of 63.4% versus 68.5% for direct and inferential images, respectively, with no significant gap. To locate this failure, we introduce OccludeTrace, a four-stage diagnostic that isolates where the visual reasoning process breaks down. The bottleneck emerges when models must recover the relevant object from the projection within the full image: they succeed on only 50% of failed examples, compared with 91% of examples they answer correctly. Extended reasoning, tool use, and few-shot prompting yield inconsistent improvements and do not reliably eliminate the gap. In contrast, an inpaint-then-confront intervention, which prompts models to first generate a candidate hidden object and then evaluate that hypothesis against the visible physical cue, reduces the gap to a statistically non-significant difference for both models with image-generation capabilities. Together, our results establish a tractable, controlled diagnostic for isolating how VLMs reason about visual content that is missing from an image.
When VLMs Meet Image Classification: Test Sets Renovation via Missing Label Identification
University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);University of Glasgow(格拉斯哥大学);Boston University(波士顿大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州))
Many Dialects, Many Languages, One Cultural Lens: Evaluating Multilingual VLMs for Bengali Culture Understanding Across Historically Linked Languages and Regional Dialects
United International University(国际联合大学);BRAC University(布拉塔克大学);University of Maryland, Baltimore County(马里兰大学巴尔的摩县分校)
Evaluating Compliance with Visualization Guidelines in Diagrams for Scientific Publications Using Large Vision Language Models
Department of Computer Science, University of Applied Sciences and Arts Dortmund(应用科学与艺术大学多特蒙德计算机科学系);Institute for Medical Informatics, Biometry and Epidemiology (IMIBE)(医学信息学、生物统计与流行病学研究所);Institute for Artificial Intelligence in Medicine (IKIM), University Hospital Essen(医学人工智能研究所,埃森大学医院)
How Do VLMs Behave When Blind or Misled? Behavioral Evaluation of VLMs on Scientific Figures
University of Aberdeen(阿伯丁大学);International Institute of Information Technology Hyderabad(海德拉巴国际信息技术学院);University of Technology Nuremberg(纽伦堡工业大学);University of Southern California(南加利福尼亚大学)
Are VLMs Ready for Autonomous Driving? An Empirical Study from the Reliability, Data, and Metric Perspectives
University of California, Irvine(加利福尼亚大学欧文分校);Shanghai AI Laboratory(上海人工智能实验室);National University of Singapore(新加坡国立大学);S-Lab, Nanyang Technological University(南洋理工大学S实验室);The University of Hong Kong(香港大学)
Seeing Symbols, Missing Cultures: Probing Vision-Language Models' Reasoning on Fire Imagery and Cultural Meaning
Duncan of Jordanstone College of Art & Design (DJCAD), University of Dundee(邓迪大学邓迪艺术与设计学院(DJCAD));Guangzhou Institute of Science and Technology (GZIST)(广州科学技术研究院);Faculty of Arts, Xiamen University(厦门大学艺术学院);University of Birmingham(伯明翰大学)
Safeguarding Vision-Language Models: Mitigating Vulnerabilities to Gaussian Noise in Perturbation-based Attacks
University of Science and Technology of China(中国科学技术大学);The Hong Kong Polytechnic University(香港理工大学);University of Washington(华盛顿大学);Nanjing University(南京大学);Stanford University(斯坦福大学);University of Chinese Academy of Sciences(中国科学院大学)
Cohere Labs Community(Cohere实验室社区);Indian Institute of Science(印度科学学院);UIUC(伊利诺伊大学厄巴纳-香槟分校);Imperial College London(帝国理工学院);Eisai Inc.(卫材公司);EleutherAI(EleutherAI组织);Georgia Institute of Technology(佐治亚理工学院)
Topo-R1: Detecting Topological Anomalies via Vision-Language Models
Stony Brook University(石溪大学);Massachusetts General Hospital and Harvard Medical School(麻省总医院和哈佛医学院);Stanford University(斯坦福大学);Penn State University(宾夕法尼亚州立大学)
Position: The Systemic Lack of Agency in Visual Reasoning
National Engineering Research Center for Multimedia Software, Institute of Artificial Intelligence, School of Computer Science, Wuhan University(武汉大学计算机学院人工智能研究所多媒体软件工程研究中心);Hubei Key Laboratory of Multimedia(湖北省多媒体与网络通信工程重点实验室);Zhongguancun Academy, Beijing, China. 100094(中关村学院,北京,中国);School of Automation, Beijing Institute of Technology(北京理工大学自动化学院);Shanda AI Research Tokyo(上海大势人工智能研究东京)
Explain Before You Answer: A Survey on Compositional Visual Reasoning
Monash University(墨尔本大学);Stanford University(斯坦福大学);University of Washington(华盛顿大学);Griffith University(格里菲斯大学);Princeton University(普林斯顿大学);Allen Institute for Artificial Intelligence(人工智能研究院)
Defining and Evaluating Visual Language Models' Basic Spatial Abilities: A Perspective from Psychometrics
School of Architecture, Tsinghua University(清华大学建筑学院);Department of Electronic Engineering, Tsinghua University(清华大学电子工程系);BNRist, Tsinghua University(清华大学BNRist)
LSceneLLM: Enhancing Large 3D Scene Understanding Using Adaptive Visual Preferences
South China University of Technology(华南理工大学);Tencent Robotics X(腾讯Robotics X实验室);Northeastern University(东北大学);UMass Amherst(马萨诸塞大学阿默斯特分校);MIT-IBM Watson AI Lab(麻省理工学院-IBM沃森人工智能实验室);Pazhou Laboratory(琶洲实验室);Sichuan University(四川大学)