TTT-Parkour: 为感知机器人攀爬实现快速测试时训练
TTT-Parkour: Rapid Test-Time Training for Perceptive Robot Parkour
AI总结:
TTT-Parkour通过快速测试时训练方法,使人形机器人能够高效适应复杂地形,提升其在极端几何结构中的攀爬能力。
摘要:
在未见过的复杂地形上实现高度动态的人形攀爬仍是一个机器人学难题。尽管通用运动策略在广泛地形分布上表现出能力,但它们往往难以应对任意且极具挑战性的环境。为克服这一限制,我们提出了一种实到仿真到实的框架,利用在新地形上的快速测试时训练(TTT),显著增强了机器人在极端几何结构中的穿越能力。我们采用两阶段端到端学习范式:首先在多样化的程序生成地形上预训练策略,然后在高保真度的从现实世界捕获中重建的网格上进行快速微调。具体而言,我们开发了一条前馈、高效且高保真的几何重建管道,使用RGB-D输入,确保在测试时训练期间的速度和质量。我们证明了TTT-Parkour使人形机器人能够掌握复杂的障碍物,包括楔形、尖刺、箱子、梯形和狭窄的梁。整个捕获、重建和测试时训练的流程在大多数测试地形上需要不到10分钟。大量实验表明,经过测试时训练的策略表现出稳健的零样本仿真到现实转移能力。
英文摘要:
Achieving highly dynamic parkour with humanoid robots on complex terrains unseen during the pre-training stage remains a challenge in robotics. Although general locomotion policies demonstrate capabilities across broad terrain distributions, they often struggle with arbitrary and highly challenging environments. To overcome this limitation, we propose a real-to-sim-to-real framework for rapid test-time training (TTT): environment-specific policy fine-tuning in simulation before deployment on a newly encountered terrain. We adopt a two-stage end-to-end learning paradigm: a policy is first pre-trained on diverse procedurally generated terrains, followed by rapid fine-tuning on high-fidelity meshes reconstructed from real-world captures. Specifically, we develop an efficient, feed-forward, and high-fidelity geometry reconstruction pipeline using RGB-D inputs, ensuring both speed and quality during test-time training. We demonstrate that TTT-Parkour empowers humanoid robots to master complex obstacles, including wedges, stakes, boxes, trapezoids, and narrow beams. The entire pipeline, including capture, reconstruction, and test-time training, takes approximately 10 minutes for most tested terrains. Experiments show improved traversal on the tested terrains after fine-tuning, with direct sim-to-real deployment and no further policy updates on the robot.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。