Robot See Robot Do：基于单目4D重建的铰接物体操作模仿
Robot See Robot Do: Imitating Articulated Object Manipulation with Monocular 4D Reconstruction
AI总结：
本研究提出RSRD方法，通过4D可微部件模型（4D-DPM）从单目RGB视频恢复3D部件运动，使双臂机器人能模仿人类演示的铰接物体操作，在90次试验中端到端成功率达60%。
摘要：
人类只需观看他人操作即可学会操作新物体；赋予机器人从这类演示中学习的能力将提供一种指定新行为的自然交互接口。本研究开发了Robot See Robot Do（RSRD），这是一种在给定单次静态多视角物体扫描的情况下，仅从单目RGB人类演示中模仿铰接物体操作的方法。我们首先提出了4D可微部件模型（4D Differentiable Part Models, 4D-DPM），这是一种利用可微渲染从单目视频中恢复3D部件运动的方法。这种“综合分析”方法在迭代优化中使用以部件为中心的特征场，从而能够利用几何正则化器仅从单一视频中恢复3D运动。基于该4D重建结果，机器人通过规划双臂运动来复现物体轨迹，从而引发所演示的物体部件运动。通过将演示表示为以部件为中心的轨迹，RSRD专注于复现演示的预期行为，同时考虑机器人自身的形态限制，而不是试图重现人手运动。我们在带有真实值标注的3D部件轨迹上评估了4D-DPM的3D追踪精度，并在双臂YuMi机器人上对9个物体各进行10次试验以评估RSRD的物理执行性能。RSRD的每个阶段平均成功率为87%，在总共90次试验中端到端总成功率为60%。值得注意的是，这是仅使用从大型预训练视觉模型中蒸馏出的特征场实现的——无需任何特定任务的训练、微调、数据集收集或标注。项目页面：https://robot-see-robot-do.github.io
英文摘要：
Humans can learn to manipulate new objects by simply watching others; providing robots with the ability to learn from such demonstrations would enable a natural interface specifying new behaviors. This work develops Robot See Robot Do (RSRD), a method for imitating articulated object manipulation from a single monocular RGB human demonstration given a single static multi-view object scan. We first propose 4D Differentiable Part Models (4D-DPM), a method for recovering 3D part motion from a monocular video with differentiable rendering. This analysis-by-synthesis approach uses part-centric feature fields in an iterative optimization which enables the use of geometric regularizers to recover 3D motions from only a single video. Given this 4D reconstruction, the robot replicates object trajectories by planning bimanual arm motions that induce the demonstrated object part motion. By representing demonstrations as part-centric trajectories, RSRD focuses on replicating the demonstration's intended behavior while considering the robot's own morphological limits, rather than attempting to reproduce the hand's motion. We evaluate 4D-DPM's 3D tracking accuracy on ground truth annotated 3D part trajectories and RSRD's physical execution performance on 9 objects across 10 trials each on a bimanual YuMi robot. Each phase of RSRD achieves an average of 87% success rate, for a total end-to-end success rate of 60% across 90 trials. Notably, this is accomplished using only feature fields distilled from large pretrained vision models -- without any task-specific training, fine-tuning, dataset collection, or annotation. Project page: https://robot-see-robot-do.github.io
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
