FreeDiff：面向扩散模型图像编辑的渐进式频率截断
FreeDiff: Progressive Frequency Truncation for Image Editing with Diffusion Models
AI总结：
针对扩散模型图像编辑中目标区域与引导影响范围错位的问题，提出无需微调的FreeDiff方法，通过渐进式频率截断优化引导，在多类编辑任务上达到与SOTA相当的效果。
摘要：
利用文本到图像模型进行精确图像编辑凭借其出色的生成能力和易用性，受到了越来越多的关注。但这类尝试在实际应用中面临一个核心挑战：预期的精确编辑目标区域与引导信号影响的更大范围之间存在错位。尽管已有研究借助注意力机制开发出了优化编辑引导的优秀方法，但这些方法需要通过复杂的网络架构进行修改，且仅适用于特定编辑任务。本研究从频率视角重新审视扩散过程与错位问题，发现由于自然图像的幂律特性和衰减噪声调度，去噪网络在早期时间步主要恢复图像的低频分量，因此会为编辑带来过多的低频信号。基于这一发现，我们提出了一种无需微调的新方法FreeDiff，该方法采用渐进式频率截断来优化扩散模型的引导，可应用于通用编辑任务。我们的方法在多种编辑任务和各类图像上均取得了与当前最优方法相当的结果，彰显了其作为图像编辑应用通用工具的潜力。
英文摘要：
Precise image editing with text-to-image models has attracted increasing interest due to their remarkable generative capabilities and user-friendly nature. However, such attempts face the pivotal challenge of misalignment between the intended precise editing target regions and the broader area impacted by the guidance in practice. Despite excellent methods leveraging attention mechanisms that have been developed to refine the editing guidance, these approaches necessitate modifications through complex network architecture and are limited to specific editing tasks. In this work, we re-examine the diffusion process and misalignment problem from a frequency perspective, revealing that, due to the power law of natural images and the decaying noise schedule, the denoising network primarily recovers low-frequency image components during the earlier timesteps and thus brings excessive low-frequency signals for editing. Leveraging this insight, we introduce a novel fine-tuning free approach that employs progressive $\textbf{Fre}$qu$\textbf{e}$ncy truncation to refine the guidance of $\textbf{Diff}$usion models for universal editing tasks ($\textbf{FreeDiff}$). Our method achieves comparable results with state-of-the-art methods across a variety of editing tasks and on a diverse set of images, highlighting its potential as a versatile tool in image editing applications.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
