Friction-Aware Safety Locomotion for Wheeled-legged Robots using Vision Language Models and Reinforcement Learning
University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Department of Electrical and Computer Engineering(电气与计算机工程系);Department of Mechanical Science Engineering(机械科学与工程系);School of Software(软件学院)
Controlling Wheeled-legged robots is challenging especially on slippery surfaces due to their dependence on continuous ground contact. Unlike quadrupeds or bipeds, which can leverage multiple fixed contact points for recovery, wheeled-legged robots are highly susceptible to slip, where even momentary loss of traction can result in irrecoverable instability. Anticipating ground physical properties such as friction before contact would allow proactive control adjustments, reducing slip risk. In this paper, we propose a friction-aware safety locomotion framework that integrates Vision-Language Models (VLMs) with a Reinforcement Learning (RL) policy. Our method employs a Retrieval-Augmented Generation (RAG) approach to estimate the Coefficient of Friction (CoF), which is then explicitly incorporated into the RL policy. This enables the robot to adapt its speed based on predicted friction conditions before contact. The framework is validated through experiments in both simulation and on a physical customized Wheeled Inverted Pendulum (WIP). Experimental results show that our approach successfully completes trajectory tracking tasks on slippery surfaces, whereas baseline methods relying solely on proprioceptive feedback fail. These findings highlight the importance and effectiveness of explicitly predicting and utilizing ground friction information for safe locomotion. They also point to a promising research direction in exploring the use of VLMs for estimating ground conditions, which remains a significant challenge for purely vision-based methods.
MUJICA: Multi-skill Unified Joint Integration of Control Architecture for Wheeled-Legged Robots
College of Intelligent Robotics and Advanced Manufacturing, Fudan University(智能机器人与先进制造学院,复旦大学);Power China Huadong Engineering Corporation Limited(中国电力工程顾问集团华东分公司)
ATRos: Learning Energy-Efficient Agile Locomotion for Wheeled-legged Robots
Shanghai Huawei Technologies Co., Ltd.(上海华为技术有限公司);College of Future Information Technology, Fudan University(复旦大学未来信息科技学院);Robotics Institute, Carnegie Mellon University(卡内基梅隆大学机器人研究所);School of Aeronautic and Astronautics, Zhejiang University(浙江大学航空宇航学院);School of Electronics and Communication Engineering, Sun Yat-sen University(中山大学电子与通信工程学院)
Unleashing the Agility of Wheeled-Legged Robots for High-Dynamic Reflexive Obstacle Evasion
School of Mechanical Engineering, Tianjin University(天津大学机械工程学院);School of Computing, National University of Singapore(新加坡国立大学计算机学院);Beijing Zhongguancun Academy(北京中关村学院);DeepRobotics(深睿科技)
ViTAL: Vision-Based Terrain-Aware Locomotion for Legged Robots
Istituto Italiano di Tecnologia (IIT)(意大利理工学院)
这项工作是基于视觉的规划策略的足式机器人分离到立足点选择和姿势适应运动规划。目前的姿态自适应策略优化机器人的身体姿势相对于给定的立足点。如果未到达这些立足点,则机器人可能最终处于没有可到达的安全立足点的状态。因此,我们提出了一个基于视觉的地形感知运动(ViTAL)的战略,包括新的姿态适应和立足点选择算法。ViTAL在姿势适应中引入了一种不同的范例,该范例不优化相对于给定立足点的身体姿势,而是最大化腿部到达安全立足点的机会的身体姿势。ViTAL根据机器人的能力和地形感知能力来规划立足点和姿势。我们使用90 kg HyQ和140 kg HyQReal四足机器人来验证ViTAL,并表明它们能够以不同的速度和步态爬上各种障碍物,包括楼梯,间隙和粗糙的地形。我们比较ViTAL与基线策略,选择机器人姿势的基础上,给定选定的立足点,并显示ViTAL优于基线。
Disturbance-Aware Adaptive Compensation in Hybrid Force-Position Locomotion Policy for Legged Robots
Department of Automation, Shanghai Jiao Tong University(自动化系,上海交通大学);Department of Computer Science, Shanghai Jiao Tong University(计算机科学系,上海交通大学);MoE Key Lab of Artificial Intelligence and AI Institute, Shanghai Jiao Tong University(人工智能与AI研究院,上海交通大学)
Istituto Italiano di Tecnologia(意大利技术研究院);Università degli Studi di Genova(热那亚大学);Swiss Federal Institute of Technology at Lausanne(洛桑联邦理工学院);University of Texas(德克萨斯大学);University of Naples Federico II(那不勒斯费德里科二世大学)
A general locomotion control framework for multi-legged locomotors
Georgia Institute of Technology(佐治亚理工学院);Emory University(埃默里大学);National University of Singapore(新加坡国立大学);Carnegie Mellon University(卡内基梅隆大学);Morehouse College(莫尔豪斯学院);Pennsylvania State University(宾夕法尼亚州立大学);Rancho Research Institute(兰乔研究所);Royal Veterinary College(皇家兽医学院);University of Oviedo(奥维耶多大学)
Adaptive Non-linear Centroidal MPC with Stability Guarantees for Robust Locomotion of Legged Robots
Artificial and Mechanical Intelligence (AMI), Istituto Italiano di Tecnologia (IIT)(人工智能与机械智能(AMI),意大利技术研究院(IIT));Dynamic Legged Systems (DLS), Istituto Italiano di Tecnologia (IIT)(动态腿部系统(DLS),意大利技术研究院(IIT));Frontier Robotics, Innovative Research Excellence(前沿机器人,创新研究卓越;本田研发,日本埼玉);Honda R&D, Saitama, Japan(机器学习与优化,曼彻斯特大学);Machine Learning and Optimisation, The University of Manchester
Soft Adaptive Feet for Legged Robots: An Open-Source Model for Locomotion Simulation
Soft Robotics for Human Cooperation and Rehabilitation, Fondazione Istituto Italiano di Tecnologia(人类合作与康复软机器人,意大利技术研究院);Humanoids and Human Centered Mechatronics, Fondazione Istituto Italiano di Tecnologia(人形与以人类为中心的机电一体化,意大利技术研究院);Department of Bioengineering, Imperial College London(生物工程系,伦敦帝国理工学院)
Dynamic Quadrupedal Legged and Aerial Locomotion via Structure Repurposing
Department of Electrical and Computer Engineering, Northeastern University(电气与计算机工程系,东北大学);Department of Aerospace Engineering, California Institute of Technology(航空航天工程系,加州理工学院)
A Foot Resistive Force Model for Legged Locomotion on Muddy Terrains
Department of Mechanical and Aerospace Engineering, Rutgers University(罗格斯大学机械与航空航天工程系);Department of Electrical and Biomedical Engineering, University of Nevada(内华达大学电气与生物医学工程系)
DRS-LIP: Linear Inverted Pendulum Model for Legged Locomotion on Dynamic Rigid Surfaces
University of Massachusetts Lowell(马萨诸塞大学洛厄尔分校);NVIDIA Research(英伟达研究);Princeton University(普林斯顿大学);Department of Mechanical Engineering, University of Massachusetts Lowell(马萨诸塞大学洛厄尔分校机械工程系)