Vision-language models have recently emerged as promising planners for autonomous driving, where success hinges on topology-aware reasoning over spatial structure and dynamic interactions from multimodal input. However, existing models are typically trained without supervision that explicitly encodes these relational dependencies, limiting their ability to infer how agents and other traffic entities influence one another from raw sensor data. In this work, we bridge this gap with a novel model-agnostic method that conditions language-based driving models on structured relational context in the form of traffic scene graphs. We serialize scene graphs at various abstraction levels and formats, and incorporate them into models via structured prompt templates, enabling systematic analysis of when and how relational supervision is most beneficial and computationally efficient. Extensive evaluations on the LangAuto and Bench2Drive benchmarks show that scene graph conditioning yields large and persistent improvements. We observe a substantial performance increase in the Driving Score of our proposed approach versus competitive LMDrive, BEVDriver, and SimLingo baselines. These results indicate that diverse architectures can effectively internalize and ground relational priors through scene graph-conditioned training, even without requiring scene graph input at test-time. Code, fine-tuned models, and our scene graph dataset are publicly available at https://github.com/iis-esslingen/GraphPilot.
Video Models as Native 4D Renderers: World-Grounded Conditioning from Animated Mesh
Tsinghua University（清华大学）；The Hong Kong Polytechnic University（香港理工大学）；University of Electronic Science and Technology of China（电子科技大学）；Sun Yat-sen University（中山大学）；The University of Hong Kong（香港大学）；University of Science and Technology of China（中国科学技术大学）；Nanyang Technological University（南洋理工大学）
Conditioning Residuals for Diffusion Models via Representation Feedback
State Key Laboratory of Virtual Reality Technology and Systems, Beihang University（虚拟现实技术与系统国家重点实验室，北京航空航天大学）；Institute of Artificial Intelligence, Beihang University（北京航空航天大学人工智能研究院）
Think Proprioceptively: State-Grounded Visual Token Selection for VLA Policies
The Hong Kong Polytechnic University（香港理工大学）；Eastern Institute of Technology（东区技术研究所）；Great Bay University（大湾大学）；Northeast Forestry University（东北林业大学）；National University of Singapore（新加坡国立大学）
Lighting-grounded Video Generation with Renderer-based Agent Reasoning
State Key Laboratory for Multimedia Information Processing, School of Computer Science, Peking University（北京大学计算机学院多媒体信息处理国家重点实验室）；National Engineering Research Center of Visual Technology, School of Computer Science, Peking University（北京大学计算机学院国家视觉技术工程研究中心）；Beijing Academy of Artificial Intelligence（北京智源人工智能研究院）；OpenBayes Information Technology Co., Ltd.（北京开贝信息技术有限公司）；School of Artificial Intelligence, Beijing University of Posts and Telecommunications（北京邮电大学人工智能学院）
FOCA: Future-Oriented Conditioning for Data-Efficient Vision-Language-Action Adaptation
Center for AI Research, VinUniversity, Vietnam；University of Utah, USA；German Research Center for Artificial Intelligence (DFKI)；Technical University of Denmark, Denmark；University of Oldenburg, Germany；University of Stuttgart, Germany；Max Planck Research School for Intelligent Systems (IMPRS-IS), Germany
FlowHOI: Flow-based Semantics-Grounded Generation of Hand-Object Interactions for Dexterous Robot Manipulation
Mohamed bin Zayed University of Artificial Intelligence（莫扎德·扎耶德人工智能大学）；Technical University of Munich（慕尼黑技术大学）；National University of Singapore（新加坡国立大学）；Westlake University（西湖大学）
University of California, Berkeley（加州大学伯克利分校）；Boston University（波士顿大学）
Solving grounded language tasks often requires reasoning about relationships between objects in the context of a given task. For example, to answer the question "What color is the mug on the plate?" we must check the color of the specific mug that satisfies the "on" relationship with respect to the plate. Recent work has proposed various methods capable of complex relational reasoning. However, most of their power is in the inference structure, while the scene is represented with simple local appearance features. In this paper, we take an alternate approach and build contextualized representat…
Renmin University of China（中国人民大学）；Northwestern Polytechnical University（西北工业大学）；Australian Centre for Robotic Vision, University of Adelaide（阿德莱德大学澳大利亚机器人视觉中心）
Humans are able to describe image contents with coarse to fine details as they wish. However, most image captioning models are intention-agnostic which can not generate diverse descriptions according to different user intentions initiatively. In this work, we propose the Abstract Scene Graph (ASG) structure to represent user intention in fine-grained level and control what and how detailed the generated description should be. The ASG is a directed graph consisting of three types of \textbf{abstract nodes} (object, attribute, relationship) grounded in the image without any concrete semantic lab…
