OneTo3D：单张图像生成可重编辑的动态3D模型与视频
OneTo3D: One Image to Re-editable Dynamic 3D Model and Video Generation
AI总结：
本文提出OneTo3D，利用Gaussian Splatting、骨架自动生成与自适应绑定及动作控制算法，从单图生成可编辑动态3D模型和语义连续、时长不限的3D视频。
摘要：
从单张图像生成可编辑的动态3D模型和视频，是单图像到3D表示或图像3D重建研究领域中的一个新方向和变革。与最初的Neural Radiance Fields（神经辐射场）相比，Gaussian Splatting（高斯泼溅）已经在隐式3D重建中展现出优势。随着技术和原理的快速发展，人们尝试使用Stable Diffusion模型，通过文本指令生成目标模型。然而，使用常规的隐式机器学习方法很难获得精确的运动和动作控制；此外，也难以生成内容较长且语义连续的3D视频。为解决这一问题，我们提出了OneTo3D，这是一种使用单张图像生成可编辑3D模型，并生成目标语义连续、时长不受限制的3D视频的方法和理论。我们使用一个常规的基础Gaussian Splatting模型从单张图像生成3D模型，这需要较少的显存和计算机计算能力。随后，我们设计了一种用于对象骨架的自动生成与自适应绑定机制。结合我们提出的可重编辑运动和动作分析与控制算法，在构建3D模型精确运动和动作控制，以及根据输入文本指令生成稳定、语义连续、时长不受限制的3D视频方面，我们能够取得优于该领域SOTA项目的性能。本文将分析详细的实现方法和理论，并给出相关比较与结论。该项目代码已开源。
英文摘要：
One image to editable dynamic 3D model and video generation is novel direction and change in the research area of single image to 3D representation or 3D reconstruction of image. Gaussian Splatting has demonstrated its advantages in implicit 3D reconstruction, compared with the original Neural Radiance Fields. As the rapid development of technologies and principles, people tried to used the Stable Diffusion models to generate targeted models with text instructions. However, using the normal implicit machine learning methods is hard to gain the precise motions and actions control, further more, it is difficult to generate a long content and semantic continuous 3D video. To address this issue, we propose the OneTo3D, a method and theory to used one single image to generate the editable 3D model and generate the targeted semantic continuous time-unlimited 3D video. We used a normal basic Gaussian Splatting model to generate the 3D model from a single image, which requires less volume of video memory and computer calculation ability. Subsequently, we designed an automatic generation and self-adaptive binding mechanism for the object armature. Combined with the re-editable motions and actions analyzing and controlling algorithm we proposed, we can achieve a better performance than the SOTA projects in the area of building the 3D model precise motions and actions control, and generating a stable semantic continuous time-unlimited 3D video with the input text instructions. Here we will analyze the detailed implementation methods and theories analyses. Relative comparisons and conclusions will be presented. The project code is open source.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
