Real2Edit2Real：通过3D控制界面生成机器人演示
Real2Edit2Real: Generating Robotic Demonstrations via a 3D Control Interface
AI总结：
本文提出Real2Edit2Real框架，通过3D可编辑性与2D视觉数据结合，减少重复数据收集，提升空间泛化能力，实验表明其数据效率提升显著。
摘要：
近年来，机器人学习的进步主要得益于大规模数据集和强大的视觉-运动策略架构，但策略鲁棒性仍受限于收集多样化演示的高昂成本，尤其在 manipulation 任务中的空间泛化。为减少重复数据收集，我们提出 Real2Edit2Real 框架，通过3D控制界面将3D可编辑性与2D视觉数据结合，生成新的演示。首先，通过多视角RGB观测重建场景几何，使用度量尺度3D重建模型。基于重建的几何，对点云进行深度可靠的3D编辑，生成新的操纵轨迹，同时通过几何校正机器人姿态以恢复物理一致的深度，作为合成新演示的可靠条件。最后，我们提出一个由深度引导的多条件视频生成模型，结合动作、边缘和射线图，合成空间增强的多视角操纵视频。在四个真实世界操纵任务上的实验表明，仅使用1-5个源演示生成的数据训练的策略可与50个真实演示数据训练的策略匹配或超越，数据效率提升最高达10-50倍。此外，高度和纹理编辑的实验结果展示了框架的灵活性和可扩展性，表明其作为统一数据生成框架的潜力。项目网站是https://real2edit2real.github.io/。
英文摘要：
Recent progress in robot learning has been driven by large-scale datasets and powerful visuomotor policy architectures, yet policy robustness remains limited by the substantial cost of collecting diverse demonstrations, particularly for spatial generalization in manipulation tasks. To reduce repetitive data collection, we present Real2Edit2Real, a framework that generates new demonstrations by bridging 3D editability with 2D visual data through a 3D control interface. Our approach first reconstructs scene geometry from multi-view RGB observations with a metric-scale 3D reconstruction model. Based on the reconstructed geometry, we perform depth-reliable 3D editing on point clouds to generate new manipulation trajectories while geometrically correcting the robot poses to recover physically consistent depth, which serves as a reliable condition for synthesizing new demonstrations. Finally, we propose a multi-conditional video generation model guided by depth as the primary control signal, together with action, edge, and ray maps, to synthesize spatially augmented multi-view manipulation videos. Experiments on four real-world manipulation tasks demonstrate that policies trained on data generated from only 1-5 source demonstrations can match or outperform those trained on 50 real-world demonstrations, improving data efficiency by up to 10-50x. Moreover, experimental results on height and texture editing demonstrate the framework's flexibility and extensibility, indicating its potential to serve as a unified data generation framework. Project website is https://real2edit2real.github.io/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
