arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2407.02873cs.RO
收藏

使用扩散模型的视频生成中机器人形状与位置保持

Robot Shape and Location Retention in Video Generation Using Diffusion Models

Manchester Metropolitan University(曼彻斯特城市大学);JD Sports Fashion PLC(JD体育时尚公司)

Peng Wang, Zhihao Guo, Abdul Latheef Sait, Minh Huy Pham

AI总结:

本文提出一种扩散模型方法,通过嵌入机器人姿态信息和在ConvNext骨干网络中施加语义掩码约束,提升视频生成中机器人形状与位置的保持能力,为碰撞检测模型提供训练数据并避免现实数据采集的法律伦理问题。

摘要:

扩散模型在图像和视频生成技术的提升方面标志着一个重要的里程碑。然而,生成能够精确保持移动物体(如机器人)形状和位置的视频仍然是一项挑战。本文提出了专门定制的扩散模型,用于生成能够准确保持移动机器人形状和位置的视频。这一进展为从事检测人与机器人之间危险交互的研究人员提供了实质性帮助,因为它有助于为碰撞检测模型创建训练数据,从而避免从现实世界收集数据,而现实世界的数据收集往往涉及法律和伦理问题。我们的模型结合了多种技术,例如嵌入可获取的机器人姿态信息,以及在ConvNext骨干网络中应用语义掩码约束。这些技术旨在细化中间输出,从而提高形状和位置的保持性能。通过大量实验,与基准扩散模型相比,我们的模型在保持不同机器人的形状和位置以及提升整体视频生成质量方面均表现出显著改进。代码将在Github上开源:https://github.com/PengPaulWang/diffusion-robots。

英文摘要:

Diffusion models have marked a significant milestone in the enhancement of image and video generation technologies. However, generating videos that precisely retain the shape and location of moving objects such as robots remains a challenge. This paper presents diffusion models specifically tailored to generate videos that accurately maintain the shape and location of mobile robots. This development offers substantial benefits to those working on detecting dangerous interactions between humans and robots by facilitating the creation of training data for collision detection models, circumventing the need for collecting data from the real world, which often involves legal and ethical issues. Our models incorporate techniques such as embedding accessible robot pose information and applying semantic mask regulation within the ConvNext backbone network. These techniques are designed to refine intermediate outputs, therefore improving the retention performance of shape and location. Through extensive experimentation, our models have demonstrated notable improvements in maintaining the shape and location of different robots, as well as enhancing overall video generation quality, compared to the benchmark diffusion model. Codes will be opensourced at \href{https://github.com/PengPaulWang/diffusion-robots}{Github}.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑