R2RDreamer: 面向空间泛化的2D操作策略的3D感知数据增强
R2RDreamer: 3D-aware Data Augmentation for Spatially-generalized 2D Manipulation Policies
AI总结：
提出R2RDreamer框架，通过轻量级3D编辑和2D视频补全，从少量真实演示生成几何一致的增强数据，提升2D操作策略的空间泛化能力。
摘要：
空间泛化对于模仿学习的操作策略至关重要，但通常需要跨不同物体姿态、机器人配置和相机视角的大规模演示。从少量源演示中进行数据增强为昂贵的真实世界数据收集提供了一种实用替代方案。基于仿真的增强可以创建可控变化，但需要复杂的环境和物体设置，并可能引入仿真到现实的差距。最近的实到实方法通过联合编辑真实演示的3D观测和动作轨迹来避免这些问题，但它们仍然依赖于强大的3D场景解析和几何补全，并且通常生成针对3D点云策略而非基于RGB的2D策略的观测。我们提出R2RDreamer，一个实到实演示增强框架，它在保持3D动作-观测编辑的几何一致性的同时，将视觉补全迁移到2D视频空间。具体来说，R2RDreamer首先通过在一个共享的3D框架中编辑不完整的物体点云和末端执行器轨迹来执行轻量级3D增强；然后，它将编辑后的场景投影到具有遮挡感知推理的掩码图像空间控制视频中，并使用密集控制图像到视频模型来补全时间上连贯的RGB观测。在空间偏移操作任务上的实验，包括2D扩散风格策略和视觉-语言-动作策略，表明R2RDreamer从有限的源演示中提高了空间泛化能力，分析验证了3D编辑、遮挡感知投影和视频补全的贡献。
英文摘要：
Spatial generalization is critical for imitation-learned manipulation policies, but achieving it typically requires scaling demonstrations across diverse object poses, robot configurations, and camera viewpoints. Data augmentation from a few source demonstrations offers a practical alternative to costly real-world collection. Simulation-based augmentation can create controllable variation, but requires complex environment and object setup and may introduce a sim-to-real gap. Recent real-to-real methods avoid these issues by jointly editing 3D observations and action trajectories from real demonstrations, yet they still rely on strong 3D scene parsing and geometry completion, and often produce observations tailored to 3D pointcloud policies rather than RGB-based 2D policies. We propose R2RDreamer, a real-to-real demonstration augmentation framework that preserves the geometric consistency of 3D action-observation editing while moving visual completion to 2D video space. Specifically, R2RDreamer first performs lightweight 3D augmentation by editing incomplete object pointclouds and end-effector trajectories in a shared 3D frame; it then projects the edited scene into masked image-space control videos with occlusion-aware reasoning and uses a dense-control image-to-video model to complete temporally coherent RGB observations. Experiments on spatially shifted manipulation tasks with both 2D diffusion-style policies and vision-language-action policies show that R2RDreamer improves spatial generalization from limited source demonstrations, with analyses validating the contributions of 3D editing, occlusion-aware projection, and video completion.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
