PhysGen：基于刚体物理的图像到视频生成
PhysGen: Rigid-Body Physics-Grounded Image-to-Video Generation
AI总结：
PhysGen结合基于模型的物理仿真与数据驱动视频生成，通过图像理解、刚体动力学仿真和视频扩散渲染三个模块，实现了物理合理且精确可控的图像到视频生成。
摘要：
我们提出了PhysGen，一种新颖的图像到视频生成方法，该方法将单张图像和输入条件（例如施加于图像中物体的力和力矩）进行转换，以生成逼真、物理上合理且时间一致的视频。我们的核心见解是将基于模型的物理仿真与数据驱动的视频生成过程相结合，从而实现合理的图像空间动力学。我们系统的核心包含三个关键组件：（i）一个图像理解模块，用于有效捕捉图像的几何形状、材质和物理参数；（ii）一个图像空间动力学仿真模型，利用刚体物理和推断出的参数来模拟逼真的行为；（iii）一个基于图像的渲染与优化模块，利用生成式视频扩散技术生成具有模拟运动的逼真视频片段。生成的视频在物理和外观上都非常逼真，甚至可以精确控制，通过定量比较和全面的用户研究，展示了其优于现有数据驱动图像到视频生成工作的结果。PhysGen生成的视频可用于各种下游应用，例如将图像转化为逼真的动画，或允许用户与图像交互并创造各种动态效果。项目页面：https://stevenlsw.github.io/physgen/
英文摘要：
We present PhysGen, a novel image-to-video generation method that converts a single image and an input condition (e.g., force and torque applied to an object in the image) to produce a realistic, physically plausible, and temporally consistent video. Our key insight is to integrate model-based physical simulation with a data-driven video generation process, enabling plausible image-space dynamics. At the heart of our system are three core components: (i) an image understanding module that effectively captures the geometry, materials, and physical parameters of the image; (ii) an image-space dynamics simulation model that utilizes rigid-body physics and inferred parameters to simulate realistic behaviors; and (iii) an image-based rendering and refinement module that leverages generative video diffusion to produce realistic video footage featuring the simulated motion. The resulting videos are realistic in both physics and appearance and are even precisely controllable, showcasing superior results over existing data-driven image-to-video generation works through quantitative comparison and comprehensive user study. PhysGen's resulting videos can be used for various downstream applications, such as turning an image into a realistic animation or allowing users to interact with the image and create various dynamics. Project page: https://stevenlsw.github.io/physgen/
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
