I2V3D：具有3D引导的可控图像到视频生成
I2V3D: Controllable image-to-video generation with 3D guidance
AI总结：
提出I2V3D框架，结合3D几何引导与生成模型，通过关键帧生成和免训练双向视频插值，实现单图到可控高质量视频的生成。
摘要：
我们提出了I2V3D，这是一种新颖的框架，可将静态图像生成为具有精确3D控制的动态视频，同时利用3D几何引导和先进生成模型的优势。我们的方法将计算机图形学流程的精确性与生成式AI的视觉保真度相结合：前者能够对相机运动、物体旋转和角色动画等元素实现准确控制，后者则可从粗略渲染的输入生成高质量视频。为支持具有任意初始起点和扩展序列的动画，我们采用由3D几何引导的两阶段生成流程：1）3D引导关键帧生成，其中定制化图像扩散模型细化渲染关键帧，以确保一致性和质量；2）3D引导视频插值，这是一种无需训练的方法，利用双向引导在关键帧之间生成平滑、高质量的视频帧。实验结果突显了我们框架通过协调3D几何与生成模型，从单张输入图像生成可控、高质量动画的有效性。我们框架的代码将公开发布。
英文摘要：
We present I2V3D, a novel framework for animating static images into dynamic videos with precise 3D control, leveraging the strengths of both 3D geometry guidance and advanced generative models. Our approach combines the precision of a computer graphics pipeline, enabling accurate control over elements such as camera movement, object rotation, and character animation, with the visual fidelity of generative AI to produce high-quality videos from coarsely rendered inputs. To support animations with any initial start point and extended sequences, we adopt a two-stage generation process guided by 3D geometry: 1) 3D-Guided Keyframe Generation, where a customized image diffusion model refines rendered keyframes to ensure consistency and quality, and 2) 3D-Guided Video Interpolation, a training-free approach that generates smooth, high-quality video frames between keyframes using bidirectional guidance. Experimental results highlight the effectiveness of our framework in producing controllable, high-quality animations from single input images by harmonizing 3D geometry with generative models. The code for our framework will be publicly released.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
