arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.05017cs.RO
收藏

直接接触容忍的运动规划与视觉语言模型

Direct Contact-Tolerant Motion Planning With Vision Language Models

State Key Laboratory of Internet of Things for Smart City (SKL-IOTSC), University of Macau(物联网智能城市国家重点实验室,澳门大学);Shenzhen Institutes of Advanced Technology (SIAT), Chinese Academy of Sciences(深圳先进技术研究院,中国科学院);Department of Electrical and Computer Engineering, The University of Hong Kong(香港大学电子与计算机工程系)

He Li, Jian Sun, Chengyang Li, Guoliang Li, Qiyu Ruan, Shuai Wang, Chengzhong Xu

AI总结:

本文提出直接接触容忍运动规划方法,利用视觉语言模型实现接触感知导航,提升机器人在拥挤环境中的鲁棒性和效率。

摘要:

在拥挤环境中导航往往需要机器人容忍与可移动或变形物体的接触以保持效率。现有接触容忍运动规划(CTMP)方法依赖于间接空间表示(例如预建地图、障碍物集),导致不准确性和对环境不确定性的适应性不足。为了解决这个问题,我们提出了一种直接接触容忍(DCT)规划器,该规划器将视觉语言模型(VLMs)整合到直接点感知和导航中,包括两个关键组件。第一个是VLM点云分割器(VPP),它利用VLM在图像空间中执行接触容忍推理,缓存推理掩码,通过里程计在帧间传播它们,并将它们投影到当前扫描以生成接触感知的点云。第二个创新是VPP引导导航(VGN),它将CTMP制定为在直接接触感知点云约束下的感知到控制优化问题,该问题进一步通过专门的深度神经网络(DNN)解决。我们在Isaac Sim和类现实汽车机器人上实现了DCT,证明DCT在有可移动障碍物的拥挤环境中实现了稳健且高效的导航,优于代表性的基线方法在多种指标上。代码可在:https://github.com/ChrisLeeUM/DCT获取。

英文摘要:

Navigation in cluttered environments often requires robots to tolerate contact with movable or deformable objects to maintain efficiency. Existing contact-tolerant motion planning (CTMP) methods rely on indirect spatial representations (e.g., prebuilt map, obstacle set), resulting in inaccuracies and a lack of adaptiveness to environmental uncertainties. To address this issue, we propose a direct contact-tolerant (DCT) planner, which integrates vision-language models (VLMs) into direct point perception and navigation, including two key components. The first one is VLM point cloud partitioner (VPP), which performs contact-tolerance reasoning in image space using VLM, caches inference masks, propagates them across frames using odometry, and projects them onto the current scan to generate a contact-aware point cloud. The second innovation is VPP guided navigation (VGN), which formulates CTMP as a perception-to-control optimization problem under direct contact-aware point cloud constraints, which is further solved by a specialized deep neural network (DNN). We implement DCT in Isaac Sim and a real car-like robot, demonstrating that DCT achieves robust and efficient navigation in cluttered environments with movable obstacles, outperforming representative baselines across diverse metrics. The code is available at: https://github.com/ChrisLeeUM/DCT.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑