XEmbodied: A Foundation Model with Enhanced Geometric and Physical Cues for Large-Scale Embodied Environments
Tsinghua University（清华大学）；Automotive and Robotics, Xiaomi Corporation（小鹏汽车与机器人部，小鹏科技）；National University of Singapore（新加坡国立大学）；McGill University（麦吉尔大学）；University of Wisconsin–Madison（威斯康星大学麦迪逊分校）
Kangan Qian, ChuChu Xie, Yang Zhong, Jingrui Pang, Siwen Jiao, Sicong Jiang, Zilin Huang, Yunlong Wang, Kun Jiang, Mengmeng Yang, Hao Ye, Guanghao Zhang, Hangju… 展开作者
Kangan Qian, ChuChu Xie, Yang Zhong, Jingrui Pang, Siwen Jiao, Sicong Jiang, Zilin Huang, Yunlong Wang, Kun Jiang, Mengmeng Yang, Hao Ye, Guanghao Zhang, Hangjun Ye, Guang Chen, Long Chen, Diange Yang
Vision-Language-Action (VLA) models drive next-generation autonomous systems, but training them requires scalable, high-quality annotations from complex environments. Current cloud pipelines rely on generic vision-language models (VLMs) that lack geometric reasoning and domain semantics due to their 2D image-text pretraining. To address this mismatch, we propose XEmbodied, a cloud-side foundation model that endows VLMs with intrinsic 3D geometric awareness and interaction with physical cues (e.g., occupancy grids, 3D boxes). Instead of treating geometry as auxiliary input, XEmbodied integrates geometric representations via a structured 3D Adapter and distills physical signals into context tokens using an Efficient Image-Embodied Adapter. Through progressive domain curriculum and reinforcement learning post-training, XEmbodied preserves general capabilities while demonstrating robust performance across 18 public benchmarks. It significantly improves spatial reasoning, traffic semantics, embodied affordance, and out-of-distribution generalization for large-scale scenario mining and embodied VQA.
University of California, Los Angeles（加州大学洛杉矶分校）；Beijing Institute for General Artificial Intelligence（北京通用人工智能研究院）；Peking University（北京大学）；Tsinghua University（清华大学）
We study the understanding of embodied reference: One agent uses both language and gesture to refer to an object to another agent in a shared physical environment. Of note, this new visual task requires understanding multimodal cues with perspective-taking to identify which object is being referred to. To tackle this problem, we introduce YouRefIt, a new crowd-sourced dataset of embodied reference collected in various physical scenes; the dataset contains 4,195 unique reference clips in 432 indoor scenes. To the best of our knowledge, this is the first embodied reference dataset that allows us…
Guide, Think, Act: Interactive Embodied Reasoning in Vision-Language-Action Models
Futian Laboratory（福田实验室）；Faculty of Computing, Harbin Institute of Technology（哈尔滨工业大学计算机学院）；International Digital Economy Academy (IDEA)（国际数字经济学院（IDEA））；School of Robotics, Hunan University（湖南大学机器人学院）；South China University of Technology（华南理工大学）；Visincept（Visincept公司）；National Key Laboratory of Smart Farm Technologies and Systems（智能农业技术与系统国家重点实验室）
Just Noticeable Difference Modeling for Token Compression in Vision-Language-Action Models
The Hong Kong Polytechnic University（香港理工大学）；Nanyang Technological University（南洋理工大学）；Bytedance Inc.（字节跳动公司）；Shandong University（山东大学）；CNRS（法国国家科学研究中心）；CentraleSupelec（中央理工学院）；Université Paris-Saclay（巴黎萨克雷大学）
GuardianBench: A Same-Scene Instruction-Contrastive Benchmark for Latent Contextual Risk in Embodied AI
Shandong University（山东大学）；National University of Singapore（新加坡国立大学）；Nanjing University of Aeronautics and Astronautics（南京航空航天大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Xiaomi Corporation（小米公司）
Out-of-Sight Embodied Agents: Multimodal Tracking, Sensor Fusion, and Trajectory Forecasting
Department of Electrical and Computer Engineering, Northeastern University（东北大学电气与计算机工程系）；Khoury College of Computer Sciences, Northeastern University（东北大学库里计算机科学学院）
A Multimodal Depth-Aware Method For Embodied Reference Understanding
Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）；KIT Campus Transfer GmbH (KCT)（KIT校园转移有限公司）；Istanbul Technical University（伊斯坦布尔技术大学）；Carnegie Mellon University（卡内基梅隆大学）
CAPE: A CLIP-Aware Pointing Ensemble of Complementary Heatmap Cues for Embodied Reference Understanding
Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）；Istanbul Technical University（伊斯坦布尔技术大学）；Carnegie Mellon University（卡内基梅隆大学）；KIT Campus Transfer GmbH (KCT)（KIT校园转移有限责任公司）
Knowing You at First Glance: Inferring Apparent Personality from Faces
Institute of Automation, CAS（中国科学院自动化研究所）；Institute of Information Engineering, CAS（中国科学院信息工程研究所）；School of Cyber Security, UCAS（中国科学院大学网络空间安全学院）；School of Artificial Intelligence, UCAS（中国科学院大学人工智能学院）；Centre for Artificial Intelligence and Robotics, Hong Kong Institute of Science & Innovation, CAS（中国科学院香港创新研究院人工智能与机器人中心）；School of Computer Science and Engineering, the Faculty of Innovation Engineering, M.U.S.T（澳门科技大学创新工程学院计算机科学与工程系）
Human-robot collaboration is an essential research topic in artificial intelligence (AI), enabling researchers to devise cognitive AI systems and affords an intuitive means for users to interact with the robot. Of note, communication plays a central role. To date, prior studies in embodied agent navigation have only demonstrated that human languages facilitate communication by instructions in natural languages. Nevertheless, a plethora of other forms of communication is left unexplored. In fact, human communication originated in gestures and oftentimes is delivered through multimodal cues, e.g…
Active Spatial Inspection for Effective and Efficient Embodied Exploration
The Hong Kong Polytechnic University（香港理工大学）；China Three Gorges University（三峡大学）；Huazhong University of Science and Technology（华中科技大学）；Shenzhen University（深圳大学）；Nanjing University of Aeronautics and Astronautics（南京航空航天大学）
TimeSpot: Benchmarking Geo-Temporal Understanding in Vision-Language Models in Real-World Settings
Computational Intelligence and Operations Laboratory (CIOL), Bangladesh（计算智能与运筹实验室（CIOL），孟加拉国）；Shahjalal University of Science and Technology (SUST), Sylhet, Bangladesh（沙赫jalal科学与技术大学（SUST），沙赫里尔，孟加拉国）；North South University (NSU), Dhaka, Bangladesh（北南大学（NSU），达卡，孟加拉国）；Qatar Computing Research Institute (QCRI), Doha, Qatar（卡塔尔计算研究中心（QCRI），多哈，卡塔尔）
Evaluating point-light biological motion in multimodal large language models
Psychiatry and Biobehavioral Sciences, UCLA（乌尔拉克大学精神病学与生物行为科学系）；Brain and Creativity Institute, USC（美国大学脑与创造力研究所）；Google DeepMind, Zurich（谷歌深度Mind瑞士分公司）
Talk2Sensors: 3D Visual Grounding in Autonomous Driving via Sensor-Adaptive Physical Cue Matching
Thrust of Artificial Intelligence, The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州）人工智能学域）；MMLab, CUHK（香港中文大学MMLab）；School of Transportation Science and Engineering, Harbin Institute of Technology（哈尔滨工业大学交通科学与工程学院）；School of Advanced Technology, Xi’an Jiaotong-Liverpool University（西交利物浦大学先进技术学院）；School of Electronics and Computer Science, University of Southampton（南安普顿大学电子与计算机科学学院）；School of Intelligent Manufacturing and Smart Transportation, Suzhou City University（苏州城市学院智能制造与智能交通学院）；Qingdao University of Science and Technology（青岛科技大学）；Institute for Math & AI, Wuhan University（武汉大学数学与人工智能研究院）；Institute of Big Data, Fudan University（复旦大学大数据研究院）
Embodied Multimodal Grounding for Open-Vocabulary Mobile Manipulation via Semantic 3D Gaussian Splatting
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Midea Group（美的集团）；The Hong Kong University of Science and Technology（香港科技大学）
Weakly-supervised VLM-guided Partial Contrastive Learning for Visual Language Navigation
University of New South Wales（新南威尔士大学）；Adobe Research（Adobe研究）；University of California San Diego（加州大学圣地亚哥分校）；Northeastern University（东北大学）；CSIRO’s Data61 and University of New South Wales（CSIRO Data61和新南威尔士大学）
University of California, Riverside（加州大学河滨分校）；Mitsubishi Electric Research Labs（三菱电机研究实验室）
Recent years have seen embodied visual navigation advance in two distinct directions: (i) in equipping the AI agent to follow natural language instructions, and (ii) in making the navigable world multimodal, e.g., audio-visual navigation. However, the real world is not only multimodal, but also often complex, and thus in spite of these advances, agents still need to understand the uncertainty in their actions and seek instructions to navigate. To this end, we present AVLEN~ -- an interactive agent for Audio-Visual-Language Embodied Navigation. Similar to audio-visual navigation tasks, the goal…
Fine-Grained Instruction-Guided Graph Reasoning for Vision-and-Language Navigation
Guangdong Institute of Intelligence Science and Technology（广东智能科学与技术研究院）；Department of Computer Science, Emory University（埃默里大学计算机科学系）；Department of Computer Science, University of Wisconsin-Madison（威斯康星大学麦迪逊分校计算机科学系）；Tsinghua University（清华大学）；Innovation and Research and Development Department, BoardWare Information System Company（BoardWare信息系统公司创新与研发部）；School of Mathematics, Peking University（北京大学数学学院）
Department of Computer Science（计算机科学系）；Delft University of Technology（代尔夫特理工大学）；LatentWorlds AI；National Policelab AI & Model-Driven Decisions Lab（国家警务实验室AI与模型驱动决策实验室）
Think Hierarchically, Act Dynamically: Hierarchical Multi-modal Fusion and Reasoning for Vision-and-Language Navigation
City University of Hong Kong, Dongguan Campus（香港城市大学东莞校区）；The University of Melbourne（墨尔本大学）；Tsinghua University（清华大学）；Baidu Inc.（百度公司）；University of Chinese Academy of Science（中国科学院大学）；National University of Singapore（新加坡国立大学）
OmniAct3D: Leveraging Foundation Geometry and Evidence-Grounded Reasoning for Panoramic 3D Detection
Hunan University（湖南大学）；National Engineering Research Center of Robot Visual Perception and Control Technology（国家机器人视觉感知与控制技术工程研究中心）；Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）
University of Chinese Academy of Sciences（中国科学院大学）
Vision-and-language navigation (VLN) is the task to enable an embodied agent to navigate to a remote location following the natural language instruction in real scenes. Most of the previous approaches utilize the entire features or object-centric features to represent navigable candidates. However, these representations are not efficient enough for an agent to perform actions to arrive the target location. As knowledge provides crucial information which is complementary to visible content, in this paper, we propose a Knowledge Enhanced Reasoning Model (KERM) to leverage knowledge to improve ag…
