arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2406.02549cs.CV
收藏

Dreamguider:改进的免训练基于扩散的条件生成

Dreamguider: Improved Training free Diffusion-based Conditional Generation

Johns Hopkins University(约翰斯·霍普金斯大学)

Nithin Gopalakrishnan Nair, Vishal M Patel

AI总结:

提出Dreamguider,一种免训练的轻量级扩散条件生成方法,通过时变因子调节梯度流、经验引导尺度及轻量级增强策略,消除了计算繁重的反向传播和手工调参,有效解决线性和非线性引导问题。

摘要:

扩散模型已成为免训练条件生成的强大工具。然而,推理时引导技术的一个关键障碍是需要通过扩散网络进行计算量庞大的反向传播来估计引导方向。此外,这些技术通常需要针对具体情况进行手工参数调优。尽管最近的一些工作为线性逆问题引入了计算量极小的方法,但仍缺乏一种同时适用于线性和非线性引导问题的通用轻量级引导解决方案。为此,我们提出了Dreamguider,这是一种无需通过扩散网络进行计算量庞大的反向传播即可实现推理时引导的方法。核心思想是通过一个时变因子来调节梯度流。此外,我们提出了一种适用于多种任务的经验引导尺度,从而消除了手工参数调优的需要。我们进一步引入了一种有效的轻量级增强策略,显著提升了推理时引导的性能。我们展示了在多个数据集和模型上使用Dreamguider进行多任务的实验,以证明所提模块的有效性。为了促进进一步研究,我们将在评审过程结束后公开代码。

英文摘要:

Diffusion models have emerged as a formidable tool for training-free conditional generation.However, a key hurdle in inference-time guidance techniques is the need for compute-heavy backpropagation through the diffusion network for estimating the guidance direction. Moreover, these techniques often require handcrafted parameter tuning on a case-by-case basis. Although some recent works have introduced minimal compute methods for linear inverse problems, a generic lightweight guidance solution to both linear and non-linear guidance problems is still missing. To this end, we propose Dreamguider, a method that enables inference-time guidance without compute-heavy backpropagation through the diffusion network. The key idea is to regulate the gradient flow through a time-varying factor. Moreover, we propose an empirical guidance scale that works for a wide variety of tasks, hence removing the need for handcrafted parameter tuning. We further introduce an effective lightweight augmentation strategy that significantly boosts the performance during inference-time guidance. We present experiments using Dreamguider on multiple tasks across multiple datasets and models to show the effectiveness of the proposed modules. To facilitate further research, we will make the code public after the review process.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑