基于边界引导的无学习扩散模型语义控制
Boundary Guided Learning-Free Semantic Control with Diffusion Models
AI总结：
提出BoundaryDiffusion方法，通过分析高维潜在空间并引导去噪轨迹至语义子空间边界，在无需学习额外网络的情况下实现冻结预训练扩散模型的高效语义控制。
摘要：
应用预训练的生成式去噪扩散模型（DDMs）进行诸如图像语义编辑等下游任务，在现有文献中通常需要对DDMs进行微调或学习辅助编辑网络。在本工作中，我们提出了BoundaryDiffusion方法，在冻结预训练DDMs且不学习任何额外网络的情况下，实现高效、有效且轻量级的语义控制。作为首批无需学习的扩散编辑工作之一，我们首先通过理论和经验分析马尔可夫链中中间高维潜在空间的概率和几何行为，寻求对其的全面理解。然后，我们提出进一步探索去噪轨迹中对编辑至关重要的步骤，该步骤刻画了预训练DDM的收敛性，并引入了一种自动搜索方法。最后，与DDMs语义行为相对较差的传统理解相反，我们证明所发现的关键潜在空间在无条件DDM中已在通用层面表现出语义子空间边界，这使我们能够通过单步操作引导去噪轨迹朝向目标边界，从而进行可控操作。我们在多种DPMs架构（DDPM、iDDPM）和数据集（CelebA、CelebA-HQ、LSUN-church、LSUN-bedroom、AFHQ-dog）及不同分辨率（64、256）上进行了大量实验，在各种任务场景（图像语义编辑、基于文本的编辑、无条件语义控制）中取得了优越或最先进的性能，证明了其有效性。
英文摘要：
Applying pre-trained generative denoising diffusion models (DDMs) for downstream tasks such as image semantic editing usually requires either fine-tuning DDMs or learning auxiliary editing networks in the existing literature. In this work, we present our BoundaryDiffusion method for efficient, effective and light-weight semantic control with frozen pre-trained DDMs, without learning any extra networks. As one of the first learning-free diffusion editing works, we start by seeking a comprehensive understanding of the intermediate high-dimensional latent spaces by theoretically and empirically analyzing their probabilistic and geometric behaviors in the Markov chain. We then propose to further explore the critical step for editing in the denoising trajectory that characterizes the convergence of a pre-trained DDM and introduce an automatic search method. Last but not least, in contrast to the conventional understanding that DDMs have relatively poor semantic behaviors, we prove that the critical latent space we found already exhibits semantic subspace boundaries at the generic level in unconditional DDMs, which allows us to do controllable manipulation by guiding the denoising trajectory towards the targeted boundary via a single-step operation. We conduct extensive experiments on multiple DPMs architectures (DDPM, iDDPM) and datasets (CelebA, CelebA-HQ, LSUN-church, LSUN-bedroom, AFHQ-dog) with different resolutions (64, 256), achieving superior or state-of-the-art performance in various task scenarios (image semantic editing, text-based editing, unconditional semantic control) to demonstrate the effectiveness.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
