Direct Contact-Tolerant Motion Planning With Vision Language Models
State Key Laboratory of Internet of Things for Smart City (SKL-IOTSC), University of Macau(物联网智能城市国家重点实验室,澳门大学);Shenzhen Institutes of Advanced Technology (SIAT), Chinese Academy of Sciences(深圳先进技术研究院,中国科学院);Department of Electrical and Computer Engineering, The University of Hong Kong(香港大学电子与计算机工程系)
Navigation in cluttered environments often requires robots to tolerate contact with movable or deformable objects to maintain efficiency. Existing contact-tolerant motion planning (CTMP) methods rely on indirect spatial representations (e.g., prebuilt map, obstacle set), resulting in inaccuracies and a lack of adaptiveness to environmental uncertainties. To address this issue, we propose a direct contact-tolerant (DCT) planner, which integrates vision-language models (VLMs) into direct point perception and navigation, including two key components. The first one is VLM point cloud partitioner (VPP), which performs contact-tolerance reasoning in image space using VLM, caches inference masks, propagates them across frames using odometry, and projects them onto the current scan to generate a contact-aware point cloud. The second innovation is VPP guided navigation (VGN), which formulates CTMP as a perception-to-control optimization problem under direct contact-aware point cloud constraints, which is further solved by a specialized deep neural network (DNN). We implement DCT in Isaac Sim and a real car-like robot, demonstrating that DCT achieves robust and efficient navigation in cluttered environments with movable obstacles, outperforming representative baselines across diverse metrics. The code is available at: https://github.com/ChrisLeeUM/DCT.
SL1M: Sparse L1-norm Minimization for contact planning on uneven terrain
The University of Edinburgh(爱丁堡大学);Ewha Womans University(梨花女子大学);LAAS-CNRS(法国国家科学研究中心分析与系统结构实验室);Université de Toulouse(图卢兹大学);University of Trento(特伦托大学)
Combining Sampling- and Gradient-based Planning for Contact-rich Manipulation
Politecnico di Milano(米兰理工大学);Istituto Dalle Molle di Studi sull’Intelligenza Artificiale (IDSIA)(达勒·莫勒人工智能研究所);Scuola Universitaria Professionale della Svizzera Italiana (SUPSI)(瑞士意大利语区应用科学大学);Università della Svizzera Italiana (USI)(瑞士意大利语区大学);Fraunhofer IPK(弗劳恩霍夫制造工程和自动化研究所)
A Unified Framework for Multi-Contact Path Planning in the Rolling Robot Systems
Geometric Mechanics and Mechatronics in Robotics (gm 2 R) Lab, School of Engineering, Cardiff University(机器人几何力学与机电学实验室,工程学院,卡迪夫大学);Information Science and Engineering Department, Ritsumeikan University(信息科学与工程系,立命馆大学)
Contact-Aware Motion Planning Among Movable Objects
Cheng Kar-Shun Robotics Institute, The Hong Kong University of Science and Technology(香港科技大学郑家纯机器人研究所);Institute of Cyber-Systems and Control, College of Control Science and Engineering, Zhejiang University(浙江大学控制科学与工程学院系统控制研究所);Huzhou Institute, Zhejiang University(浙江大学湖州研究院)
A Predefined-Time Convergent and Noise-Tolerant Zeroing Neural Network Model for Time Variant Quadratic Programming With Application to Robot Motion Planning
Multi-Scale Medical Robotics Center(多尺度医学机器人中心);The Chinese University of Hong Kong(香港中文大学);School of Engineering Technology, Purdue University(普渡大学工程技术学院)
Distilling Contact Planning for Fast Trajectory Optimization in Robot Air Hockey
Idiap Research Institute(Idiap研究 institute);École Polytechnique Fédérale de Lausanne(瑞士联邦理工学院);Intelligent Autonomous Systems, TU Darmstadt(智能自主系统,德累斯顿技术大学);German Research Center for AI (DFKI)(德国人工智能研究中心);Centre for Cognitive Science, Hessian.AI(认知科学中心,Hessian.AI)
Modelling and Path Planning of Snake Robot in cluttered environment
Visvesvaraya National Institute of Technology(维斯瓦拉亚国家理工学院);Carnegie Mellon University(卡内基梅隆大学);Inbitorobotics(Inbitorobotics公司);Robotics Institute, Carnegie Mellon University(卡内基梅隆大学机器人研究所)
Simultaneous Contact Sequence and Patch Planning for Dynamic Locomotion
Munich Institute of Robotics and Machine Intelligence (MIRMI), Technical University of Munich (TUM), Germany(慕尼黑机器人与机器智能研究所(MIRMI)、慕尼黑技术大学(TUM));Tandon School of Engineering, New York University (NYU), USA(纽约大学Tandon工程学院)
Learning feasible transitions for efficient contact planning
Mohamed bin Zayed University of Artificial Intelligence (MBZUAI)(穆罕默德·本·扎耶德人工智能大学(MBZUAI));Munich Institute of Robotics and Machine Intelligence, Technical University of Munich(慕尼黑工业大学慕尼黑机器人与机器智能研究所)
Bi-Level Optimization for Contact and Motion Planning in Rope-Assisted Legged Robots
Dipartimento di Ingegneria e Scienza dell’Informazione (DISI), University of Trento(工程与信息科学系(DISI),特伦托大学);Inria, Université de Lorraine, CNRS(法国国家信息与自动化技术研究院(Inria)、洛林大学、法国国家科学研究中心(CNRS))
Mediating between Contact Feasibility and Robustness of Trajectory Optimization through Chance Complementarity Constraints
Georgia Institute of Technology(佐治亚理工学院);George W. Woodruff School of Mechanical Engineering(乔治·W·伍德拉夫机械工程学院);Department of Electrical and Computer Engineering(电气与计算机工程系)
Contact-aware Path Planning for Autonomous Neuroendovascular Navigation
FuTURE Lab, Department of Robotics Engineering, Worcester Polytechnic Insitute (WPI)(沃思彻斯特理工学院未来实验室);Department of Computer Science, University of North Carolina at Chapel Hill(北卡罗来纳大学教堂山分校计算机科学系);UMass Memorial Medical Center(马萨诸塞州纪念医疗中心)
Hierarchical Planning of Dynamic Movements without Scheduled Contact Sequences
Department of Advanced Robotics, Istituto Italiano di Tecnologia(意大利技术研究院先进机器人部);Robot Learning and Interaction Group, Idiap Research Institute(伊迪普研究所机器人学习与交互小组)