VisualMimic：通过运动跟踪与生成实现类人机器人的视觉移动操作
VisualMimic: Visual Humanoid Loco-Manipulation via Motion Tracking and Generation
AI总结：
针对非结构化环境中类人机器人移动操作的感知与控制整合难题，提出VisualMimic框架，结合第一视角视觉与分层全身控制，实现模拟到现实的零样本迁移，完成多任务并泛化至户外环境。
摘要：
非结构化环境中的类人机器人移动操作需要将第一视角感知与全身控制紧密结合。然而，现有方法要么依赖外部运动捕捉系统，要么无法在不同任务间泛化。我们提出VisualMimic，一个将第一视角视觉与分层全身控制相结合的视觉模拟到现实框架，适用于类人机器人。VisualMimic将通过师生方案从人类运动数据训练的任务无关低层级关键点跟踪器，与从视觉和本体感受输入生成关键点指令的任务特定高层级策略相结合。为确保稳定训练，我们向低层级策略注入噪声，并利用人类运动统计数据裁剪高层级动作。VisualMimic实现了在模拟中训练的视觉运动策略向真实类人机器人的零样本迁移，完成了搬箱子、推箱子、足球运球和踢球等多种移动操作任务。除受控实验室环境外，我们的策略在户外环境中也具有鲁棒的泛化能力。视频可在https://visualmimic.github.io获取。
英文摘要：
Humanoid loco-manipulation in unstructured environments demands tight integration of egocentric perception and whole-body control. However, existing approaches either depend on external motion capture systems or fail to generalize across diverse tasks. We introduce VisualMimic, a visual sim-to-real framework that unifies egocentric vision with hierarchical whole-body control for humanoid robots. VisualMimic combines a task-agnostic low-level keypoint tracker -- trained from human motion data via a teacher-student scheme -- with a task-specific high-level policy that generates keypoint commands from visual and proprioceptive input. To ensure stable training, we inject noise into the low-level policy and clip high-level actions using human motion statistics. VisualMimic enables zero-shot transfer of visuomotor policies trained in simulation to real humanoid robots, accomplishing a wide range of loco-manipulation tasks such as box lifting, pushing, football dribbling, and kicking. Beyond controlled laboratory settings, our policies also generalize robustly to outdoor environments. Videos are available at: https://visualmimic.github.io .
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
