Existing vision-language-action (VLA) models act in 3D real-world but are typically built on 2D encoders, leaving a spatial reasoning gap that limits generalization and adaptability. Recent 3D integration techniques for VLAs either require specialized sensors and transfer poorly across modalities, or inject weak cues that lack geometry and degrade vision-language alignment. In this work, we introduce FALCON (From Spatial to Action), a novel paradigm that injects rich 3D spatial tokens into the action head. FALCON leverages spatial foundation models to deliver strong geometric priors from RGB alone, and includes an Embodied Spatial Model that can optionally fuse depth, or pose for higher fidelity when available, without retraining or architectural changes. To preserve language reasoning, spatial tokens are consumed by a Spatial-Enhanced Action Head rather than being concatenated into the vision-language backbone. These designs enable FALCON to address limitations in spatial representation, modality transferability, and alignment. In comprehensive evaluations across three simulation benchmarks and eleven real-world tasks, our proposed FALCON achieves state-of-the-art performance, consistently surpasses competitive baselines, and remains robust under clutter, spatial-prompt conditioning, and variations in object scale and height.
GroundingPI: A Grounding Foundation Model towards Physical Intelligence with Visual Primitives
XPeng Inc.(小鹏汽车);Peking University(北京大学);The University of Hong Kong(香港大学);University of California, Berkeley(加州大学伯克利分校);Princeton University(普林斯顿大学);National University of Singapore(新加坡国立大学);Tsinghua University(清华大学);HKUST (GZ)(香港科技大学(广州))
Grounding Foundational Vision Models with 3D Human Poses for Robust Action Recognition
Arizona State University(亚利桑那州立大学);Emory University(埃默里大学);University of California, Berkeley(加州大学伯克利分校);Delft University of Technology(代尔夫特理工大学);Algoverse AI Research(Algoverse AI 研究)
Clutter-Robust Vision-Language-Action Models through Object-Centric and Geometry Grounding
University of Arkansas(阿拉巴马大学);National University of Singapore(新加坡国立大学);TU Wien(维也纳技术大学);Max Planck Research School for Intelligent Systems(智能系统马克斯·普朗克研究学校);University of Stuttgart(斯图加特大学);University of Liverpool(利物浦大学)
WildRoadBench: A Wild Aerial Road-Damage Grounding Benchmark for Vision-Language Models and Autonomous Agents
University of Electronic Science and Technology of China(电子科技大学);National University of Singapore(新加坡国立大学);De Artificial Intelligence Lab(德人工智能实验室);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));University of Science and Technology of China(中国科学技术大学)
Grounding Actions in Camera Space: Observation-Centric Vision-Language-Action Policy
College of Computer Science and Technology, Zhejiang University(浙江大学计算机科学与技术学院);Shanghai AI Lab(上海人工智能实验室);SenseTime Research(商汤科技研究院);Nanjing University(南京大学);Tsinghua University(清华大学)
College of Intelligence and Computing, Tianjin University(天津大学智能与计算学部)
Visual context provides grounding information for multimodal machine translation (MMT). However, previous MMT models and probing studies on visual features suggest that visual information is less explored in MMT as it is often redundant to textual information. In this paper, we propose an object-level visual context modeling framework (OVC) to efficiently capture and explore visual information for multimodal machine translation. With detected objects, the proposed OVC encourages MMT to ground translation on desirable visual objects by masking irrelevant objects in the visual modality. We equip…
LocateAnything: Fast and High-Quality Vision-Language Grounding with Parallel Box Decoding
The Hong Kong Polytechnic University(香港理工大学);Princeton University(普林斯顿大学);Nanjing University(南京大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校)
From Region Arrival to Instance-Level Grounding in Vision-and-Language Navigation
Australian Institute for Machine Learning, Adelaide University(澳大利亚机器学习研究所,阿德莱德大学);Ruyi Dynamics(如意动力);Fudan University(复旦大学);Swiss Federal Institute of Technology Lausanne (EPFL)(瑞士洛桑联邦理工学院)
Adaptive Articulated Object Manipulation On The Fly with Foundation Model Reasoning and Part Grounding
Beijing University of Posts and Telecommunications(北京邮电大学);School of Computer Science, Peking University(北京大学计算机学院);School of EECS, Peking University(北京大学电子工程学院)
Seoul National University(首尔国立大学);Hanyang University(汉阳大学)
The main challenge in vision-and-language navigation (VLN) is how to understand natural-language instructions in an unseen environment. The main limitation of conventional VLN algorithms is that if an action is mistaken, the agent fails to follow the instructions or explores unnecessary regions, leading the agent to an irrecoverable path. To tackle this problem, we propose Meta-Explore, a hierarchical navigation method deploying an exploitation policy to correct misled recent actions. We show that an exploitation policy, which moves the agent toward a well-chosen local goal among unvisited but…
WaterVG: Waterway Visual Grounding based on Text-Guided Vision and mmWave Radar
Institute of Deep Perception Tech, JITRI(JITRI深度感知技术研究所);School of Advanced Technology, Xi’an Jiaotong-Liverpool University(西交利物浦大学先进技术学院);Faculty of Science and Engineering, University of Liverpool(利物浦大学科学与工程学院);Thrust of Artificial Intelligence and Thrust of Intelligent Transportation, HKUST (Guangzhou)(香港科技大学(广州)人工智能学科及智能交通学科)
GG-Nav: Goal-oriented Grounding Chain-of-Thought Elicits Reasoning in Navigation Foundation Models
Zhejiang University(浙江大学);Shanghai AI Laboratory(上海人工智能实验室);University of Science and Technology of China(中国科学技术大学);Tongji University(同济大学);The University of Hong Kong(香港大学)
Representation-Aligned Tactile Grounding for Contact-Rich Robotic Manipulation
Shanghai Key Lab of Intelligent Information Processing, College of Computer Science and Artificial Intelligence, Fudan University(复旦大学计算机科学与人工智能学院智能信息处理上海市重点实验室);College of Intelligent Robotics and Advanced Manufacturing, Fudan University(复旦大学智能机器人与先进制造学院);Lenovo CTO Organization(联想首席技术官组织);Nanyang Technological University(南洋理工大学);TeleAl, China Telecom(中国电信天翼人工智能公司)
Talk2Escape: Conversational Grounding for Vision-and-Language Navigation
Australian Institute for Machine Learning, Adelaide University(阿德莱德大学澳大利亚机器学习研究所);Zhejiang Wanli University(浙江万里学院);Fudan University(复旦大学);Agency for Science, Technology and Research (A*STAR)(新加坡科技研究局)