RoboSwap：一种用于无监督机械臂替换的GAN驱动的视频扩散框架
RoboSwap: A GAN-driven Video Diffusion Framework For Unsupervised Robot Arm Swapping
AI总结：
针对机器人学习中跨具身数据稀缺的问题，提出无监督框架RoboSwap，通过结合GAN与扩散模型，在非配对数据上实现机械臂替换，提升了视频结构连贯性与运动一致性。
摘要：
生成模型的最新进展彻底改变了视频合成与编辑。然而，高质量且多样化的数据集的稀缺性继续阻碍着基于视频条件的机器人学习，限制了跨平台泛化能力。在这项工作中，我们解决了将一个视频中的机械臂替换为另一个视频中的机械臂的挑战：这是跨具身学习的关键步骤。与以前依赖于在相同环境设置下的成对视频演示的方法不同，我们提出的框架RoboSwap在来自不同环境的非配对数据上运行，减轻了数据收集的需求。RoboSwap引入了一种新颖的视频编辑流程，集成了GAN和扩散模型，结合了它们各自的优势。具体来说，我们将机械臂从其背景中分割出来，并训练一个非配对GAN模型将一种机械臂转换为另一种。转换后的机械臂与原始视频背景混合，并使用扩散模型进行细化，以增强连贯性、运动真实感和对象交互。GAN和扩散阶段是独立训练的。我们的实验表明，RoboSwap在三个基准上在结构连贯性和运动一致性方面均优于最先进的视频和图像编辑模型，从而为在机器人学习中生成可靠的跨具身数据提供了一个稳健的解决方案。
英文摘要：
Recent advancements in generative models have revolutionized video synthesis and editing. However, the scarcity of diverse, high-quality datasets continues to hinder video-conditioned robotic learning, limiting cross-platform generalization. In this work, we address the challenge of swapping a robotic arm in one video with another: a key step for crossembodiment learning. Unlike previous methods that depend on paired video demonstrations in the same environmental settings, our proposed framework, RoboSwap, operates on unpaired data from diverse environments, alleviating the data collection needs. RoboSwap introduces a novel video editing pipeline integrating both GANs and diffusion models, combining their isolated advantages. Specifically, we segment robotic arms from their backgrounds and train an unpaired GAN model to translate one robotic arm to another. The translated arm is blended with the original video background and refined with a diffusion model to enhance coherence, motion realism and object interaction. The GAN and diffusion stages are trained independently. Our experiments demonstrate that RoboSwap outperforms state-of-the-art video and image editing models on three benchmarks in terms of both structural coherence and motion consistency, thereby offering a robust solution for generating reliable, cross-embodiment data in robotic learning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
