arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2411.01212cs.CVcs.AIcs.GRcs.LG
收藏

无限分辨率积分噪声扭曲用于扩散模型

Infinite-Resolution Integral Noise Warping for Diffusion Models

Netflix(奈飞公司);Stanford University(斯坦福大学);Eyeline Studios(伊莱恩工作室);Stony Brook University(石溪大学)

Yitong Deng, Winnie Lin, Lingxiao Li, Dmitriy Smirnov, Ryan Burgert, Ning Yu, Vincent Dedun, Mohammad H. Taghavi

AI总结:

提出一种无限分辨率积分噪声扭曲算法,通过收集多个布朗桥增量,在保持分布的同时大幅降低计算成本,实现高效的时间一致视频生成,并可扩展至三维空间。

摘要:

将预训练的图像扩散模型适配为生成时间上一致的视频已成为一个具有影响力的生成建模研究方向。免训练的空间噪声操作已被证明是一种有效技术,其挑战在于在加入时间一致性的同时保持高斯白噪声分布。最近,Chang等人(2024)使用具有分布保持保证的积分噪声表示来表述该问题,并提出了一种基于上采样的算法来计算它。然而,尽管其数学表述具有优势,该算法却产生了高昂的计算成本。通过分析其算法在上采样分辨率趋于无穷大时的极限行为,我们开发了一种替代算法,该算法通过收集多个布朗桥的增量,实现了其无限分辨率的精度,同时将计算成本降低了数个数量级。我们证明并实验验证了我们的理论主张,并展示了我们方法在真实世界应用中的有效性。我们进一步表明,我们的方法可轻松扩展到三维空间。

英文摘要:

Adapting pretrained image-based diffusion models to generate temporally consistent videos has become an impactful generative modeling research direction. Training-free noise-space manipulation has proven to be an effective technique, where the challenge is to preserve the Gaussian white noise distribution while adding in temporal consistency. Recently, Chang et al. (2024) formulated this problem using an integral noise representation with distribution-preserving guarantees, and proposed an upsampling-based algorithm to compute it. However, while their mathematical formulation is advantageous, the algorithm incurs a high computational cost. Through analyzing the limiting-case behavior of their algorithm as the upsampling resolution goes to infinity, we develop an alternative algorithm that, by gathering increments of multiple Brownian bridges, achieves their infinite-resolution accuracy while simultaneously reducing the computational cost by orders of magnitude. We prove and experimentally validate our theoretical claims, and demonstrate our method's effectiveness in real-world applications. We further show that our method readily extends to the 3-dimensional space.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑