PartEdit：使用预训练扩散模型的细粒度图像编辑
PartEdit: Fine-Grained Image Editing using Pre-Trained Diffusion Models
AI总结：
提出首个基于预训练扩散模型的对象部位细粒度图像编辑方法，通过学习特殊文本令牌生成定位掩码，结合特征融合与自适应阈值策略实现精准编辑，在自建基准上全面超越现有方法。
摘要：
我们提出了首个基于文本的图像编辑方法，用于基于预训练扩散模型编辑对象部位。基于扩散的图像编辑方法利用扩散模型对图像语义的深刻理解来执行各种编辑。然而，现有的扩散模型对许多对象部位缺乏足够的理解，阻碍了用户请求的细粒度编辑。为了解决这个问题，我们提出扩展预训练扩散模型的知识，使其能够理解各种对象部位，从而执行细粒度编辑。我们通过高效的令牌优化过程学习对应不同对象部位的特殊文本令牌来实现这一点。这些令牌经过优化，在每个推理步骤中生成可靠的定位掩码以定位编辑区域。利用这些掩码，我们设计了特征融合和自适应阈值策略以无缝执行编辑。为了评估我们的方法，我们建立了一个部位编辑基准和评估协议。实验表明，我们的方法在所有指标上均优于现有编辑方法，并在进行的用户研究中66-90%的时间被用户偏好。
英文摘要：
We present the first text-based image editing approach for object parts based on pre-trained diffusion models. Diffusion-based image editing approaches capitalized on the deep understanding of diffusion models of image semantics to perform a variety of edits. However, existing diffusion models lack sufficient understanding of many object parts, hindering fine-grained edits requested by users. To address this, we propose to expand the knowledge of pre-trained diffusion models to allow them to understand various object parts, enabling them to perform fine-grained edits. We achieve this by learning special textual tokens that correspond to different object parts through an efficient token optimization process. These tokens are optimized to produce reliable localization masks at each inference step to localize the editing region. Leveraging these masks, we design feature-blending and adaptive thresholding strategies to execute the edits seamlessly. To evaluate our approach, we establish a benchmark and an evaluation protocol for part editing. Experiments show that our approach outperforms existing editing methods on all metrics and is preferred by users 66-90% of the time in conducted user studies.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
