RoboPaint：从人类示范到任何机器人和任何视角
RoboPaint: From Human Demonstration to Any Robot and Any View
AI总结：
RoboPaint通过真实-仿真-真实数据流程，将人类示范转化为机器人训练数据，实现高效、低成本的复杂灵巧操作训练。
摘要：
获取大规模、高保真的机器人示范数据仍然是扩展视觉-语言-动作（VLA）模型在灵巧操作中的关键瓶颈。我们提出了一种真实-仿真-真实的数据收集和数据编辑流程，该流程将人类示范转换为机器人可执行的、环境特定的训练数据，而无需直接的机器人远程操作。标准化的数据收集房间被构建起来，以捕捉多模态的人类示范（同步3个RGB-D视频，11个RGB视频，29自由度手套关节角度，和14通道触觉信号）。基于这些人类示范，我们介绍了一种触觉感知的重定向方法，该方法通过几何和力引导的优化将人类手状态映射到机器人dex-hand状态。然后，重定向的机器人轨迹在逼真的Isaac Sim环境中渲染，以构建机器人训练数据。现实世界实验已证明：（1）重定向的dex-hand轨迹在10种不同的物体操作任务中实现了84%的成功率。（2）仅使用我们生成的数据训练的VLA策略（Pi0.5）在三个代表性任务上平均成功率为80%，即拾取和放置、推动和倾倒。最后，我们的真实-仿真-真实数据流程可以高效地“绘制”出机器人训练数据，从人类示范中。我们提供了一种可扩展、成本效益高且性能损失小的替代方案，用于复杂灵巧操作的远程操作。
英文摘要：
Acquiring large-scale, high-fidelity robot demonstration data remains a critical bottleneck for scaling Vision-Language-Action (VLA) models in dexterous manipulation. We propose a Real-Sim-Real data collection and data editing pipeline that transforms human demonstrations into robot-executable, environment-specific training data without direct robot teleoperation. Standardized data collection rooms are built to capture multimodal human demonstrations (synchronized 3 RGB-D videos, 11 RGB videos, 29-DoF glove joint angles, and 14-channel tactile signals). Based on these human demonstrations, we introduce a tactile-aware retargeting method that maps human hand states to robot dex-hand states via geometry and force-guided optimization. Then the retargeted robot trajectories are rendered in a photorealistic Isaac Sim environment to build robot training data. Real world experiments have demonstrated: (1) The retargeted dex-hand trajectories achieve an 84\% success rate across 10 diverse object manipulation tasks. (2) VLA policies (Pi0.5) trained exclusively on our generated data achieve 80\% average success rate on three representative tasks, i.e., pick-and-place, pushing and pouring. To conclude, robot training data can be efficiently "painted" from human demonstrations using our real-sim-real data pipeline. We offer a scalable, cost-effective alternative to teleoperation with minimal performance loss for complex dexterous manipulation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
