异步补丁扩散:空间灵活的图像生成
AsyncPatch Diffusion: spatially-flexible image generation
AI总结:
提出AsyncPatch Diffusion框架,通过为不同空间区域分配不同噪声水平实现异质去噪轨迹,在保持生成质量的同时原生支持图像修复和自适应生成。
摘要:
标准扩散模型使用单一共享噪声水平破坏整个样本,迫使所有空间区域遵循相同的去噪轨迹。我们引入了AsyncPatch Diffusion,一个联合扩散框架,为不同的输入维度(如图像像素或潜在令牌)分配不同的噪声水平。我们展示了这种异步破坏如何定义有效的生成过程,同时支持更丰富的空间异质去噪轨迹,并为此过程证明了第一个有效的ELBO。我们表明,单个预训练模型可以执行空间自适应生成,其中不同区域按不同调度去噪。一个关键挑战是训练:天真的独立噪声水平采样过度强调高度异质的配置,而低估了在采样过程中至关重要的同质噪声水平。我们通过一个受控的噪声水平采样器来解决这个问题,该采样器调节平均破坏水平及其空间变异性。AsyncPatch在ImageNet 256和LSUN上实现了与常规扩散相当的生成质量,同时原生适用于图像修复而无需特定任务微调。我们进一步引入了输入引导,利用干净或部分损坏的区域来指导未知区域的生成,提高了局部一致性和纹理匹配。最后,我们展示了自适应生成策略,包括不确定性引导加速和自回归采样。
英文摘要:
Standard diffusion models corrupt an entire sample with a single shared noise level, forcing all spatial regions to follow the same denoising trajectory. We introduce AsyncPatch Diffusion, a joint-diffusion framework that assigns distinct noise levels to different input dimensions, such as image pixels, or latent tokens. We show how this asynchronous corruption defines a valid generative process while supporting a richer family of spatially heterogeneous denoising trajectories, and prove the first valid ELBO for this process. We show that a single pretrained model can perform spatially adaptive generation, where different regions are denoised on different schedules. A key challenge is training: naive independent noise-level sampling overemphasizes highly heterogeneous configurations and underrepresents homogeneous noise levels, that are crucial during sampling. We address this with a controlled noise-level sampler that regulates both the average corruption level and its spatial variability. AsyncPatch achieves generation quality comparable to conventional diffusion on ImageNet 256 and LSUN, while being natively suited for inpainting without task-specific fine-tuning. We further introduce input guidance, which uses clean or partially corrupted regions to guide the generation of unknown regions, improving local consistency and texture matching. Finally, we demonstrate adaptive generation strategies including uncertainty-guided acceleration and autoregressive sampling.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。