SAKI：从人类视频中学习技能组装与运动学模仿以实现长时程移动操作
SAKI: Skill Assembly and Kinematic Imitation from Human Videos for Long-Horizon Mobile Manipulation
浏览论文内容
中文总结 AI 辅助
提出SAKI框架，从人类视频学习可复用物体中心技能，通过跨示范组装与全身运动学模仿，实现长时程移动操作，真实机器人实验验证技能复用与组合能力。
中文摘要 AI 辅助
从人类视频中学习为获取多样化操作技能提供了一条有前景的途径。将这一能力从桌面场景扩展到长时程移动操作，需要适应并组合跨变化场景和机器人配置的示范交互。我们提出了技能组装与运动学模仿（SAKI），这是一个连接人类视频技能获取、跨示范组装和闭环全身执行的框架。SAKI准备了可复用的以物体为中心的技能，这些技能保留了任务关键交互，同时允许迁移路径进行适应。给定一个目标和提供的任务依赖关系，它选择并排序技能，将其物体角色绑定到当前场景，并在连续技能之间传递场景估计和机器人配置。全身运动学模仿生成协调的基座、手臂和夹爪运动。在执行过程中，持续的物体估计维持跨视角变化的任务参考，而视觉反馈更新剩余的轨迹。真实机器人实验展示了技能在不同布局中的复用，以及将独立演示的交互组合成连续移动任务（包括整理和擦拭）的能力。消融实验结果表明，任务条件化的参考准备在保持全身优化和视觉反馈不变的情况下，显著提高了长时程任务的完成率。请访问此URL查看视频演示：https://thisurl
英文摘要
Learning from human videos offers a promising route to acquiring diverse manipulation skills. Extending this capability beyond tabletop settings to long-horizon mobile manipulation requires adapting and composing demonstrated interactions across changing scenes and robot configurations. We present Skill Assembly and Kinematic Imitation (SAKI), a framework connecting human-video skill acquisition, cross-demonstration assembly and closed-loop whole-body execution. SAKI prepares reusable object-centric skills that preserve task-critical interactions while allowing transfer paths to adapt. Given a goal and supplied task dependencies, it selects and orders skills, binds their object roles to the current scene, and carries scene estimates and robot configuration between successive skills. Whole-body kinematic imitation generates coordinated base, arm and gripper motion. During execution, persistent object estimates maintain task references across viewpoint changes, while visual feedback updates remaining trajectories. Real-robot experiments demonstrate skill reuse across layouts and the composition of independently demonstrated interactions into continuous mobile tasks, including tidying and wiping. Ablation results show that task-conditioned reference preparation substantially improves long-horizon task completion with whole-body optimisation and visual feedback held fixed. Check https://aus.bot/research/saki/ for video demos!
发表机构
- Australian Centre for Robotics, The University of Sydney（悉尼大学澳大利亚机器人中心）
机构由 AI 辅助整理，请以论文原文为准。
