ROPA: 合成机器人姿态生成用于RGB-D双臂数据增强
ROPA: Synthetic Robot Pose Generation for RGB-D Bimanual Data Augmentation
AI总结：
本文提出ROPA，通过微调Stable Diffusion生成RGB-D双臂任务的合成数据，提升机器人双臂操控的鲁棒性与数据增强效率。
摘要：
通过模仿学习训练鲁棒的双臂操控策略需要涵盖广泛机器人姿态、接触和场景上下文的演示数据。然而，收集多样且精确的真实世界演示成本高且耗时，限制了可扩展性。先前工作通常针对单目（腕部相机）设置或生成无配对动作的新型图像，而针对第三视角（眼到手）RGB-D训练的新动作标签数据增强研究较少。本文提出ROPA，一种离线模仿学习数据增强方法，通过微调Stable Diffusion合成第三视角RGB和RGB-D观测数据。我们的方法同时生成对应的关节空间动作标签，并通过约束优化强制物理一致性，通过适当的机械臂到物体接触约束确保双臂场景的物理一致性。我们在5个模拟和3个真实任务上评估了我们的方法。在2625次模拟试验和300次真实世界试验中，我们的结果表明ROPA优于基线和消融实验，展示了其在眼到手双臂操控中可扩展的RGB和RGB-D数据增强潜力。我们的项目网站可用：https://ropaaug.github.io/.
英文摘要：
Training robust bimanual manipulation policies via imitation learning requires demonstration data with broad coverage over robot poses, contacts, and scene contexts. However, collecting diverse and precise real-world demonstrations is costly and time-consuming, which hinders scalability. Prior works have addressed this with data augmentation, typically for either eye-in-hand (wrist camera) setups with RGB inputs or for generating novel images without paired actions, leaving augmentation for eye-to-hand (third-person) RGB-D training with new action labels less explored. In this paper, we propose Synthetic Robot Pose Generation for RGB-D Bimanual Data Augmentation (ROPA), an offline imitation learning data augmentation method that fine-tunes Stable Diffusion to synthesize third-person RGB and RGB-D observations of novel robot poses. Our approach simultaneously generates corresponding joint-space action labels while employing constrained optimization to enforce physical consistency through appropriate gripper-to-object contact constraints in bimanual scenarios. We evaluate our method on 5 simulated and 3 real-world tasks. Our results across 2625 simulation trials and 300 real-world trials demonstrate that ROPA outperforms baselines and ablations, showing its potential for scalable RGB and RGB-D data augmentation in eye-to-hand bimanual manipulation. Our project website is available at: https://ropaaug.github.io/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
