Spatial reasoning poses a particular challenge for intelligent agents and is at the same time a prerequisite for their successful interaction and communication in the physical world. One such reasoning task is to describe the position of a target object with respect to the intrinsic orientation of some reference object via relative directions. In this paper, we introduce GRiD-A-3D, a novel diagnostic visual question-answering (VQA) dataset based on abstract objects. Our dataset allows for a fine-grained analysis of end-to-end VQA models' capabilities to ground relative directions. At the same time, model training requires considerably fewer computational resources compared with existing datasets, yet yields a comparable or even higher performance. Along with the new dataset, we provide a thorough evaluation based on two widely known end-to-end VQA architectures trained on GRiD-A-3D. We demonstrate that within a few epochs, the subtasks required to reason over relative directions, such as recognizing and locating objects in a scene and estimating their intrinsic orientations, are learned in the order in which relative directions are intuitively processed.
CoTBox-TTT: Grounding Medical VQA with Visual Chain-of-Thought Boxes During Test-time Training
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Data Science, The Chinese University of Hong Kong, Shenzhen（香港大学深圳研究院数据科学学院）
Beyond Accuracy: Evaluating Visual Grounding In Multimodal Medical Reasoning
The University of Texas MD Anderson Cancer Center（德克萨斯大学MD安德森癌症中心）；Cohere Labs（Cohere实验室）；Eisai Inc.（艾伯维公司）；Indian Institute of Science, Bangalore（班加罗尔印度科学研究院）
Scalable Training of Spatially Grounded 2D Vision-Language Models for Radiology
Computer Vision Group, University of Freiburg, Germany（德国弗莱堡大学计算机视觉组）；Department of Radiology, Medical Center -- University of Freiburg, Germany（德国弗莱堡大学医学中心放射科）；CRIION-AI Lab, Freiburg, Germany（德国弗莱堡CRIION-AI实验室）
Do Medical Vision Language Models Actually See? A Counterfactual Grounding Framework and Hard-Negative Contrastive Training for Visually-Reliant Medical VLMs
The University of Texas MD Anderson Cancer Center（德克萨斯大学MD安德森癌症中心）；Eisai Inc.（卫材株式会社）；IISc, Bangalore（印度科学研究所班加罗尔分校）；Cohere Labs Community（Cohere实验室社区）
