arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.18647cs.LGcs.AIcs.CVcs.ITmath.IT
收藏

噪声调度作为扩散训练中的信息引导分配

Noise Scheduling as Information-Guided Allocation in Diffusion Training

Tilburg University & JADS(蒂尔堡大学及JADS);Sony AI(索尼人工智能);University of Cambridge(剑桥大学);Radboud University(拉德堡德大学);Sony Group Corporation(索尼集团公司)

Gabriel Raya, Bac Nguyen, Georgios Batzolis, Yuhta Takida, Dejan Stancevic, Naoki Murata, Chieh-Hsin Lai, Yuki Mitsufuji, Luca Ambrogioni

AI总结:

提出InfoNoise,一种在线自适应噪声调度方法,通过估计条件熵率剖面动态调整训练噪声分布,以优化去噪任务中的信息增益,在图像、DNA和语言生成等任务中达到或超越基线,并节省高达3倍训练计算量。

摘要:

我们引入了InfoNoise,一种用于扩散训练的在线自适应噪声调度,它将优化努力重新分配到去噪最具信息量的噪声水平上。与损失加权一起,噪声调度在去噪问题之间诱导出有效的分配,而这种分配通常在知道信息性噪声水平之前就已固定。InfoNoise通过从训练期间的去噪损失中估计条件熵率剖面,使这种分配具有数据自适应性,无需辅助模型或离线搜索。通过I--MMSE,该剖面识别出噪声观测在何处能快速减少关于干净样本的不确定性,并指导训练噪声分布的适应。它只改变这个分布,保持目标、加权和参数化不变。在图像基准测试中,调度已被广泛调整,InfoNoise匹配或略微超过强基线,并且可以用更少的更新达到相同的质量。在表示、序列和模态转换(包括DNA和语言生成)上,InfoNoise优于固定和自适应基线,并且达到目标质量所需的训练计算量最多减少3倍。这些结果确立了条件熵率剖面作为噪声调度设计的数据依赖目标,并使在线自适应成为手动调度搜索的实用替代方案。

英文摘要:

We introduce InfoNoise, an online adaptive noise schedule for diffusion training that reallocates optimization effort toward noise levels where denoising is most informative. Together with loss weighting, a noise schedule induces an effective allocation across denoising problems, often fixed before informative noise levels are known. InfoNoise makes this allocation data-adaptive by estimating a conditional-entropy-rate profile from denoising losses during training, without auxiliary models or offline search. Through I--MMSE, this profile identifies where noisy observations rapidly reduce uncertainty about the clean sample and guides adaptation of the training noise distribution. It changes only this distribution, keeping the objective, weighting, and parameterization fixed. On image benchmarks, where schedules have been extensively tuned, InfoNoise matches or slightly exceeds strong baselines and can reach the same quality with fewer updates. On representation, sequence, and modality shifts, including DNA and language generation, InfoNoise improves over fixed and adaptive baselines and reaches target quality with up to $3\times$ less training compute. These results establish the conditional-entropy-rate profile as the data-dependent target for noise schedule design and make online adaptation a practical alternative to manual schedule search.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑