arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2511.20280cs.CV

通过VLM引导的迭代自优化提升物理导向的视频生成

Bootstrapping Physics-Grounded Video Generation through VLM-Guided Iterative Self-Refinement

School of Computer Science and Technology, University of Chinese Academy of Sciences(中国科学院大学计算机科学与技术学院);School of Computer Science and Technology, Beijing Institute of Technology(北京理工大学计算机科学与技术学院);Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络安全学院);Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所)

Yang Liu, Xilin Zhao, Peisong Wen, Siran Dai, Qingming Huang

AI总结:

本文提出一种通过VLM引导的迭代自优化方法,提升视频生成的物理一致性,实验显示在PhyIQ基准上得分提升明显。

摘要:

近年来视频生成在视觉质量上取得了显著进展,但当前模型仍难以生成符合现实物理原理的结果。为此,我们提出了一种迭代自优化框架,利用大语言模型和视觉语言模型为视频生成提供物理感知的指导。具体而言,我们引入了一种多模态链式思考(MM-CoT)过程,根据物理不一致性的反馈来优化提示,逐步提升生成质量。该方法无需训练且易于使用,可广泛应用于各种视频生成模型。在PhyIQ基准测试中,我们的方法将Physics-IQ分数从56.31提升到62.38。我们希望这项工作能为物理一致的视频生成提供初步探索,并为未来研究提供见解。

英文摘要:

Recent progress in video generation has led to impressive visual quality, yet current models still struggle to produce results that align with real-world physical principles. To this end, we propose an iterative self-refinement framework that leverages large language models and vision-language models to provide physics-aware guidance for video generation. Specifically, we introduce a multimodal chain-of-thought (MM-CoT) process that refines prompts based on feedback from physical inconsistencies, progressively enhancing generation quality. This method is training-free and plug-and-play, making it readily applicable to a wide range of video generation models. Experiments on the PhyIQ benchmark show that our method improves the Physics-IQ score from 56.31 to 62.38. We hope this work serves as a preliminary exploration of physics-consistent video generation and may offer insights for future research.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑