PIXELS：基于渐进式图像范例的潜空间手术编辑方法
PIXELS: Progressive Image Xemplar-based Editing with Latent Surgery
AI总结：
针对现有图像编辑方法控制粒度粗、依赖训练、兼容性差的问题，提出PIXELS框架，仅靠推理即可利用任意数量范例实现细粒度编辑，性能与用户体验均有提升。
摘要：
当前语言引导的扩散模型在图像编辑领域的进展往往受限于繁琐的提示词工程，用户难以精准表述期望的修改效果。一种更直观的替代方案是借助真实场景中的图像范例提供引导，帮助用户将想象中的编辑效果落地。现有的基于范例的编辑方法并未充分利用预训练大型文本到图像（TTI）模型所学习到的丰富潜空间，反而退而求其次，通过精心设计的目标函数进行训练来完成任务。这类方法虽然有一定效果，但需要消耗大量计算资源，且无法兼容多种基础模型，也不支持任意数量的范例输入。进一步研究发现，这些技术还限制了用户的控制能力，仅能对整个编辑区域施加统一的全局修改。本文提出了一种基于现成扩散模型的渐进式范例驱动编辑新框架，命名为PIXELS，该框架支持像素级或区域级的细粒度编辑控制，实现定制化编辑。我们的方法仅在推理阶段运行即可实现模仿式编辑，用户可从数量动态变化的参考图像或多模态提示中汲取灵感，渐进式整合所有期望的修改，无需对现有TTI模型进行重新训练或微调。这种细粒度控制能力开辟了诸多新的应用可能，包括对单个物体的选择性修改，以及指定渐进式的空间变化效果。实验证明，PIXELS能够高效生成高质量编辑结果，在量化指标和人工评估中均取得显著提升。通过降低高质量图像编辑的门槛，PIXELS有望让更广泛的用户群体借助任意开源图像生成模型，轻松实现专业级的编辑效果。
英文摘要：
Recent advancements in language-guided diffusion models for image editing are often bottle-necked by cumbersome prompt engineering to precisely articulate desired changes. An intuitive alternative calls on guidance from in-the-wild image exemplars to help users bring their imagined edits to life. Contemporary exemplar-based editing methods shy away from leveraging the rich latent space learnt by pre-existing large text-to-image (TTI) models and fall back on training with curated objective functions to achieve the task. Though somewhat effective, this demands significant computational resources and lacks compatibility with diverse base models and arbitrary exemplar count. On further investigation, we also find that these techniques restrict user control to only applying uniform global changes over the entire edited region. In this paper, we introduce a novel framework for progressive exemplar-driven editing with off-the-shelf diffusion models, dubbed PIXELS, to enable customization by providing granular control over edits, allowing adjustments at the pixel or region level. Our method operates solely during inference to facilitate imitative editing, enabling users to draw inspiration from a dynamic number of reference images, or multimodal prompts, and progressively incorporate all the desired changes without retraining or fine-tuning existing TTI models. This capability of fine-grained control opens up a range of new possibilities, including selective modification of individual objects and specifying gradual spatial changes. We demonstrate that PIXELS delivers high-quality edits efficiently, leading to a notable improvement in quantitative metrics as well as human evaluation. By making high-quality image editing more accessible, PIXELS has the potential to enable professional-grade edits to a wider audience with the ease of using any open-source image generation model.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
