Despite recent advances in multimodal content generation enabled by vision-language models (VLMs), their ability to reason about and generate structured 3D scenes remains largely underexplored. This limitation constrains their utility in spatially grounded tasks such as embodied AI, immersive simulations, and interactive 3D applications. We introduce a new paradigm that enables VLMs to generate, understand, and edit complex 3D environments by injecting a continually evolving spatial context. Constructed from multimodal input, this context consists of three components: a scene portrait that provides a high-level semantic blueprint, a semantically labeled point cloud capturing object-level geometry, and a scene hypergraph that encodes rich spatial relationships, including unary, binary, and higher-order constraints. Together, these components provide the VLM with a structured, geometry-aware working memory that integrates its inherent multimodal reasoning capabilities with structured 3D understanding for effective spatial reasoning. Building on this foundation, we develop an agentic 3D scene generation pipeline in which the VLM iteratively reads from and updates the spatial context. The pipeline features high-quality asset generation with geometric restoration, environment setup with automatic verification, and ergonomic adjustment guided by the scene hypergraph. Experiments show that our framework can handle diverse and challenging inputs, achieving a level of generalization not observed in prior work. Further results demonstrate that injecting spatial context enables VLMs to perform downstream tasks such as interactive scene editing and path planning, suggesting strong potential for spatially intelligent systems in computer graphics, 3D vision, and embodied applications. Project page: https://spatctxvlm.github.io/project_page/.
Contextualized Visual Personalization in Vision-Language Models
Department of Electrical and Computer Engineering, Seoul National University, Seoul, South Korea(电气电子工程系,首尔国立大学,首尔,韩国);Interdisciplinary Program in Artificial Intelligence, Seoul National University, Seoul, Korea(人工智能交叉学科项目,首尔国立大学,首尔,韩国)
Scaling Agentic Reinforcement Learning for Tool-Integrated Reasoning in VLMs
Virginia Tech(弗吉尼亚理工大学);Emory University(埃默里大学);KAUST(阿联酋卡塔尔大学);Georgia Tech(佐治亚理工学院);Cisco(思科系统);TAMU(德克萨斯大学奥斯汀分校);UT Southwestern Medical Center(德克萨斯西南医学中心);Eigen AI
Seeing What Is Actually There: PriVE-Bench and PriVE-Tools for Counterfactual Evaluation of Agentic Visual Evidence in VLMs
The University of Manchester(曼彻斯特大学);The University of Melbourne(墨尔本大学);University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);University of Edinburgh(爱丁堡大学);University of Southern California(南加州大学);Fudan University(复旦大学);University of Newcastle(纽卡斯尔大学)
The Eye of Sherlock Holmes: Uncovering User Private Attribute Profiling via Vision-Language Model Agentic Framework
Nanyang Technological University(南洋理工大学);Beijing University of Technology(北京理工大学);Hengxin Tech(恒心科技);Alibaba Group(阿里巴巴集团);Squirrel Ai Learning(squirrel AI 学习)
From Segments to Scenes: Temporal Understanding for Agentic Autonomous Driving via Vision-Language Models
University of California, Berkeley(加州大学伯克利分校);University of Cambridge(剑桥大学);University of Toronto(多伦多大学);ETH Zurich(苏黎世联邦理工学院);University of Washington(华盛顿大学);University of Southern California(南加州大学)
Thinking with Imagination: Agentic Visual Spatial Reasoning with World Simulators
The University of Hong Kong(香港大学);Shanghai AI Laboratory(上海人工智能实验室);Shanghai Jiao Tong University(上海交通大学);Fudan University(复旦大学);Beihang University(北航大学)
VisMem: Latent Vision Memory Unlocks Potential of Vision-Language Models
National University of Singapore(新加坡国立大学);Fudan University(复旦大学);Tsinghua University(清华大学);Zhejiang University(浙江大学);University of Science and Technology of China(中国科学技术大学);vivo
CLUTCH: Contextualized Language model for Unlocking Text-Conditioned Hand motion modelling in the wild
Technical University of Darmstadt(德累斯顿技术大学);Max-Planck Institute for Intelligent Systems(马克斯·普朗克智能系统研究所);University of Tuebingen(图宾根大学);Tubingen AI Center(图宾根人工智能中心);Max Planck Institute for Informatics(马克斯·普朗克信息研究所)
Enhancing Agentic Autonomous Scientific Discovery with Vision-Language Model Capabilities
Department of Physics, University of Cambridge, Cambridge, United Kingdom(剑桥大学物理系);Kavli Institute for Cosmology, University of Cambridge, Cambridge, United Kingdom(剑桥大学卡弗利天文研究所);Department of Physics and Astronomy, Haverford College, 370 Lancaster Avenue, Haverford, PA 19041, USA(哈弗福德学院物理与天文学系);Division of Physics, Mathematics and Astronomy, California Institute of Technology, Pasadena, CA 91125, USA(加州理工学院物理、数学与天文学系);Institute of Astronomy, University of Cambridge, Cambridge, United Kingdom(剑桥大学天文研究所)
Are Neuro-Inspired Multi-Modal Vision-Language Models Resilient to Membership Inference Privacy Leakage?
The University of Alabama(阿拉巴马大学);Alabama Center for the Advancement of AI(阿拉巴马人工智能 advancement 中心);Trustworthy AI Lab(可信人工智能实验室);Department of Computer Science, The University of Alabama(计算机科学系)