NoiseShift: 为更好的低分辨率图像生成的分辨率感知噪声校准
NoiseShift: Resolution-Aware Noise Recalibration for Better Low-Resolution Image Generation
AI总结:
本文提出NoiseShift,通过校准噪声条件以恢复局部正反向一致性,提升低分辨率图像生成质量,无需额外计算开销。
摘要:
文本到图像扩散模型在采样分辨率超出最终训练分辨率时往往性能下降。本文提出NoiseShift,通过重新索引去噪器的噪声条件,保持原始噪声采样计划不变,从而减少训练-测试不匹配,提升低分辨率生成质量。在Stable Diffusion 3、Stable Diffusion 3.5和Flux-Dev上应用后,低分辨率生成质量显著提升,FID分数分别从203降至171、310降至277以及120降至113。
英文摘要:
Text-to-image diffusion models often degrade when sampled at resolutions outside the final training resolution set. Prior work has largely emphasized higher resolution generation, enabling pretrained diffusion models to extrapolate beyond the resolutions seen during training. In this work, we instead target lower-resolution generation, performing inference at reduced resolution to significantly cut computational cost. We show that network conditioning of the noise level induces a train-test mismatch that directly degrades low-resolution generation: the same scheduled noise level can correspond to a different perceptual corruption level at lower resolutions, mis-calibrating the denoiser timestep and noise embedding. To this end, we propose NoiseShift, a training-free recalibration method that keeps the original noise sampling schedule unchanged and instead re-indexes the noise conditioning of the denoiser to restore local forward-reverse consistency. Using a lightweight coarse-to-fine calibration on a small set of image-text pairs, NoiseShift learns a resolution-specific mapping from scheduler noise to conditioning noise, reducing train-test mismatch and improving lower-resolution generation quality. When NoiseShift is applied to Stable Diffusion 3 (SD3), Stable Diffusion 3.5 (SD3.5), and Flux-Dev, generation quality at low resolutions improves consistently. Particularly, SD3 generation at 128x128 resolution gets an improved FID score from 203 to 171, and SD3.5 gets an improved FID score from 310 to 277 on LAION-COCO. Even Flux-Dev which already implements a complementary time-shifting strategy gets a modest boost from NoiseShift with an improved FID score from 120 to 113 at 64x64 resolution. More importantly, NoiseShift achieves such improvements with minimal implementation changes and no additional inference overhead.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。