arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2603.22091cs.CV
收藏

P-Flow:提示视觉效果生成

P-Flow: Prompting Visual Effects Generation

Show Lab, National University of Singapore(新加坡国立大学Show实验室)

Rui Zhao, Mike Zheng Shou

AI总结:

P-Flow通过测试时提示优化,基于参考视频与生成输出的视觉效果差异,迭代改进提示,实现高保真且多样化的动态视觉效果定制。

摘要:

近年来,视频生成模型在遵循文本提示方面取得了显著进步。然而,动态视觉效果(如物体破碎或爆炸等时间演化且外观驱动的视觉现象)的定制仍待探索。现有运动定制或控制方法主要关注低层次主体或摄像机运动,可通过显式控制信号(如运动轨迹)引导。相比之下,动态视觉效果涉及更高层次语义,更适合通过文本提示控制。但人类难以快速创作准确指定这些效果的单个提示,因其需复杂的时间推理和迭代优化。为此,我们提出P-Flow,一种无需训练的框架,用于视频生成中的动态视觉效果定制,不修改底层模型。通过利用视觉语言模型的语义和时间推理能力,P-Flow在测试时优化提示,根据参考视频与生成输出的视觉效果差异进行提示优化。通过迭代优化,提示逐步适应新场景,以更好地诱导所需动态效果。实验表明,P-Flow在文本到视频和图像到视频生成任务中均优于其他模型。代码可在https://github.com/showlab/P-Flow获取。

英文摘要:

Recent advancements in video generation models have significantly improved their ability to follow text prompts. However, the customization of dynamic visual effects, defined as temporally evolving and appearance-driven visual phenomena like object crushing or explosion, remains underexplored. Prior works on motion customization or control mainly focus on low-level motions of the subject or camera, which can be guided using explicit control signals such as motion trajectories. In contrast, dynamic visual effects involve higher-level semantics that are more naturally suited for control via text prompts. However, it is hard and time-consuming for humans to craft a single prompt that accurately specifies these effects, as they require complex temporal reasoning and iterative refinement over time. To address this challenge, we propose P-Flow, a novel training-free framework for customizing dynamic visual effects in video generation without modifying the underlying model. By leveraging the semantic and temporal reasoning capabilities of vision-language models, P-Flow performs test-time prompt optimization, refining prompts based on the discrepancy between the visual effects of the reference video and the generated output. Through iterative refinement, the prompts evolve to better induce the desired dynamic effect in novel scenes. Experiments demonstrate that P-Flow achieves high-fidelity and diverse visual effect customization and outperforms other models on both text-to-video and image-to-video generation tasks. Code is available at https://github.com/showlab/P-Flow.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑