arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2610.22347cs.CV

用于高效自回归视频生成的推测强制(Speculative Forcing)

Speculative Forcing for Efficient Autoregressive Video Generation

KAIST AI(韩国科学技术院AI学院);NAVER AI Lab(NAVER AI实验室)

Hyunwook Choi, Seungryong Kim, Dongyoon Han, Sangdoo Yun

AI总结:

该研究提出Speculative Forcing框架,通过提前接受无需进一步细化的token减少自回归视频生成的去噪步骤,在保持生成质量的同时,于LongLive和Lingbot-World 2.0等数据集上实现了显著的速度提升。

摘要:

少步自回归视频生成器支持流式生成,但无论每个token需要多少细化,都会将其通过最终去噪步骤。我们发现这种统一计算通常是不必要的:许多token在初始步骤中已与最终全步输出非常接近,而其他token则需要细化至最后一步。我们提出Speculative Forcing框架,将中间干净估计视为草稿,并预测何时可提前安全接受单个token。一旦被接受,token将被冻结为最终输出,不再进行进一步去噪,但其缓存的键和值仍可作为剩余token的上下文。我们探索两种替代验证器设计:一种是基于与干净内容的相似度及跨步骤稳定性的无训练验证器,另一种是读取冻结生成器中间特征以估计每个token与全步输出剩余差距的学习型验证器。在短视频、长视频及世界生成任务中,Speculative Forcing在保持全步采样质量的同时大幅降低计算量。在LongLive数据集上,它与四步生成器性能匹配,每个token约需1.7次去噪步骤,以41.7和37.8 fps运行时速度提升超1.5倍;在相机运动持续改变可见内容的Lingbot-World 2.0上,它以每个token约2.5次去噪步骤保持质量,以24.5 fps运行时速度提升1.35倍。

英文摘要:

Few-step autoregressive video generators enable streaming generation, yet they run every token through the final denoising step, regardless of how much refinement it needs. We find this uniform computation is often unnecessary: many tokens already closely match their eventual full-step outputs in the initial steps, while others require refinement until the end. We introduce Speculative Forcing, a framework that treats intermediate clean estimates as drafts and predicts when individual tokens can be safely accepted early. Once accepted, a token is frozen as its final output and undergoes no further denoising, while its cached keys and values remain available as context for the remaining tokens. We explore two alternative verifier designs: a training-free one based on similarity to clean content and stability across steps, and a learned one that reads the frozen generator's intermediate features to estimate each token's remaining gap to its full-step output. Across short-video, long-video, and world generation, Speculative Forcing preserves the quality of full-step sampling while substantially reducing computation. On LongLive, it matches the four-step generator with ~1.7 denoising steps per token, running over 1.5x faster at 41.7 and 37.8 fps. On Lingbot-World 2.0, where camera motion continually changes the visible content, it maintains quality with ~2.5 denoising steps per token and runs 1.35x faster at 24.5 fps.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑