arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2610.24965cs.CV
收藏

视频生成模型的元逆向提示

Meta Inverse Prompting for Video Generative Models

Korea University(高丽大学)

Yerin Jung, Jongheon Jeong

AI总结:

本研究提出元视频逆向提示(MVIP)框架,将逆向提示形式化为元学习问题,实现高效的视频逆向提示,在提升重建保真度的同时大幅提高效率。

摘要:

近期视频生成模型的进展已实现灵活的条件生成与编辑流程,例如文本到视频、图像到视频及参考引导合成。然而,这些工作流程本质上是单向的,将条件映射到视频,却未提供逆过程机制。在复杂媒体制作场景中,从给定视频恢复可控文本条件的能力极具价值,但仍未得到充分探索。尽管逆向提示已在图像生成中被研究,将这些方法直接扩展到视频颇具挑战,因为需要捕捉超越静态外观的时间动态,且逐视频优化的计算成本极高。本研究提出首个实用的高效视频逆向提示框架。具体而言,我们将逆向提示形式化为元学习问题,并引入元视频逆向提示(MVIP),其学习在时间维度上优化的元提示,以实现快速有效的提示推理。我们的方法摊销了逆过程,消除了测试时昂贵的迭代优化需求。实验结果表明,与基于图像的逆向和迭代字幕基线相比,MVIP恢复的逆向提示质量更高,重建保真度显著提升,同时效率大幅提高。代码和视频结果可在该https URL获取。

英文摘要:

Recent advances in video generative models have enabled flexible conditional generation and editing pipelines, such as text-to-video, image-to-video, and reference-guided synthesis. However, these workflows are inherently one-directional, mapping conditions to videos without providing a mechanism to invert this process. In complex media production settings, the ability to recover controllable textual conditions from a given video is highly desirable, yet remains underexplored. While inverse prompting has been studied for image generation, directly extending these approaches to video is challenging due to the need to capture temporal dynamics beyond static appearance, as well as the substantial computational cost of per-video optimization. In this work, we propose the first practical framework for efficient video inverse prompting. Specifically, we formulate inverse prompting as a meta-learning problem and introduce Meta Video Inverse Prompting (MVIP), which learns a meta-prompt optimized over temporal dimensions to enable fast and effective prompt inference. Our approach amortizes the inversion process, eliminating the need for costly iterative optimization at test time. Experimental results demonstrate that MVIP recovers higher-quality inverse prompts with significantly improved reconstruction fidelity compared to image-based inversion and iterative captioning baselines, while achieving substantial gains in efficiency. Code and video results are available at https://ueriniuno.github.io/mvip-page/.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑