PhysVLM: Enabling Visual Language Models to Understand Robotic Physical Reachability
School of Traffic and Transportation, Beijing Jiaotong University(北京交通大学交通运输学院);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);ObjectEye Inc.(奥森科技(ObjectEye公司));Guangdong Polytechnic Normal University(广东技术师范大学)
Weijie Zhou, Manli Tao, Chaoyang Zhao, Haiyun Guo, Honghui Dong, Ming Tang, Jinqiao Wang
Understanding the environment and a robot's physical reachability is crucial for task execution. While state-of-the-art vision-language models (VLMs) excel in environmental perception, they often generate inaccurate or impractical responses in embodied visual reasoning tasks due to a lack of understanding of robotic physical reachability. To address this issue, we propose a unified representation of physical reachability across diverse robots, i.e., Space-Physical Reachability Map (S-P Map), and PhysVLM, a vision-language model that integrates this reachability information into visual reasoning. Specifically, the S-P Map abstracts a robot's physical reachability into a generalized spatial representation, independent of specific robot configurations, allowing the model to focus on reachability features rather than robot-specific parameters. Subsequently, PhysVLM extends traditional VLM architectures by incorporating an additional feature encoder to process the S-P Map, enabling the model to reason about physical reachability without compromising its general vision-language capabilities. To train and evaluate PhysVLM, we constructed a large-scale multi-robot dataset, Phys100K, and a challenging benchmark, EQA-phys, which includes tasks for six different robots in both simulated and real-world environments. Experimental results demonstrate that PhysVLM outperforms existing models, achieving a 14\% improvement over GPT-4o on EQA-phys and surpassing advanced embodied VLMs such as RoboMamba and SpatialVLM on the RoboVQA-val and OpenEQA benchmarks. Additionally, the S-P Map shows strong compatibility with various VLMs, and its integration into GPT-4o-mini yields a 7.1\% performance improvement.
Prismatic-Bending Transformable (PBT) Joint for a Modular, Foldable Manipulator with Enhanced Reachability and Dexterity
Department of Mechanical Engineering, University of California, Berkeley(加州大学伯克利分校机械工程系);The Department of Mechanical and Automation Engineering, The Chinese University of Hong Kong(香港中文大学机械与自动化工程系);FANUC Advanced Research Laboratory, FANUC America Corporation, USA(FANUC美国公司先进研究实验室)
RM4D: A Combined Reachability and Inverse Reachability Map for Common 6-/7-axis Robot Arms by Dimensionality Reduction to 4D
Aston University(阿斯顿大学);Aston Centre for Artificial Intelligence Research and Application (ACAIRA)(阿斯顿人工智能研究与应用中心);Department of Applied AI & Robotics(应用人工智能与机器人学系)
Automated Generation of MDPs Using Logic Programming and LLMs for Robotic Applications
Department of Information Engineering and Computer Science, University of Trento(信息工程与计算机科学系,特伦托大学);Department of Industrial Engineering, University of Trento(工业工程系,特伦托大学)
PRTS: A Primitive Reasoning and Tasking System via Contrastive Representations
Institute of Artificial Intelligence (TeleAI), China Telecom(人工智能研究院(TeleAI),中国电信);Tsinghua University(清华大学);Shanghai Jiao Tong University(上海交通大学);Fudan University(复旦大学)
GSAM: A Generalizable and Safe Robotic Framework for Articulated Object Manipulation
College of Computer Science, Chongqing University, Chongqing, China(重庆大学计算机学院);Lumos Robotics, China(Lumos机器人中国);Xi'an Jiaotong-Liverpool University, China(西安交通大学利物浦大学);Fudan University, China(复旦大学);Department of Information Engineering and Computer Science, University of Trento, Trento, Italy(特伦托大学信息工程与计算机科学系)
Dexterous Control of an 11-DOF Redundant Robot for CT-Guided Needle Insertion With Task-Oriented Weighted Policies
Jacobs School of Engineering, University of California San Diego(加州大学圣地亚哥分校 Jacobs 工程学院);School of Medicine, University of California San Diego(加州大学圣地亚哥分校医学院);School of Medicine, University of Missouri-Kansas City(密苏里大学堪萨斯城分校医学院)
Using Mobile AR for Rapid Feasibility Analysis for Deployment of Robots: A Usability Study with Non-Expert Users
Faculty of Engineering, University of Southern Denmark(南部丹麦大学工程学院);Universal Robots A/S(Universal Robots公司);Department of Engineering, University of Cambridge(剑桥大学工程学院)