BIFE:更好的交互,更少的错误,用于分钟级视频生成
BIFE: Better Interaction, Fewer Errors for Minute-Long Video Generation
AI总结:
提出BIFE框架,通过语义稀疏KV缓存和块强制训练策略,解决长视频生成中的长程交互保持和误差累积问题,实现稳定连贯的分钟级视频生成,在VDE指标上提升约20%。
摘要:
长视频生成是构建逼真世界模型的关键步骤,需要高视觉保真度和长程交互一致性。最近的自回归扩散模型通过KV缓存重用实现了长时程生成,但面临两个基本挑战:由于滑动窗口KV缓存无法保持长程交互,以及误差累积随时间逐渐降低生成质量。为解决这些问题,我们提出BIFE框架,引入语义稀疏KV缓存用于基于检索的长程条件控制,以及块强制训练策略以强制跨块一致性。这些设计共同保留了历史交互并减轻了漂移,从而实现了稳定连贯的分钟级视频生成。我们还引入了InterVBench,一个具有细粒度块级注释和视频漂移误差指标的分钟级视频基准。在InterVBench和VBench-Long上的大量实验表明,BIFE达到了最先进的性能,包括在VDE-Subject上比基线提高22.2%,在VDE-Clarity上提高19.4%。网站:此 https URL。代码:此 https URL。
英文摘要:
Long video generation is a critical step toward building realistic world models, requiring both high visual fidelity and long-range interaction consistency. Recent autoregressive diffusion models enable long-horizon generation through KV cache reuse, yet suffer from two fundamental challenges: failure to preserve long-range interactions due to sliding-window KV cache and error accumulation that progressively degrades generation quality over time. To address these issues, we propose BIFE, a framework that introduces a semantic sparse KV cache for retrieval-based long-range conditioning and a Block Forcing training strategy to enforce cross-block consistency. Together, these designs preserve historical interactions while mitigating drift, enabling stable and coherent minute-long video generation. We also introduce InterVBench, a minute-long video benchmark with fine-grained block-level annotations and Video Drift Error metrics. Extensive experiments on InterVBench and VBench-Long demonstrate that BIFE achieves state-of-the-art performance, including a 22.2% improvement on VDE-Subject and a 19.4% improvement on VDE-Clarity over baselines. Website: https://alibaba-damo-academy.github.io/BIFE. Code: https://github.com/alibaba-damo-academy/BIFE.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。