重新思考涂鸦引导的图像编辑：泛化、指令遵循与多任务
Rethinking Scribble-Guided Image Editing: Generalization, Instruction Adherence, and Multi-Tasking
AI总结：
针对涂鸦引导图像编辑在多任务场景下性能不稳定的问题，通过实证研究揭示指令级泛化瓶颈，提出覆盖-真实课程、多任务拼接和编辑聚焦损失三种策略，在VIBE基准上实现单任务和多任务的最优结果。
摘要：
涂鸦引导的图像编辑允许用户将简单的涂鸦注释与文本提示相结合，以指定图像编辑的位置和方式，从而实现灵活交互和精确的空间控制。然而，现有模型在这种范式下仍表现出不稳定的性能，尤其是在多任务场景中。为了提升性能，我们使用开源编辑模型进行实证研究，并揭示了泛化中的不对称性：指令级泛化（包括跨编辑任务以及从单任务到多任务设置）比图像域泛化（例如从合成图像到真实图像，或从马赛克图像到常规图像）更具挑战性。这表明主要瓶颈在于对多样化编辑指令的学习不足，而非图像域差异。受此启发，我们提出了三种策略：(a) 覆盖-真实课程，一个两阶段流程，首先构建大规模合成、指令丰富的数据以提供广泛的任务监督，然后精选少量真实数据以细化生成的真实性；(b) 多任务拼接，通过几乎零成本地拼接单任务样本来构建多任务训练样本，同时使学习到的能力泛化到非马赛克图像；(c) 编辑聚焦损失，利用合成数据中输入和输出图像之间的变化区域，将训练聚焦于编辑区域，提高学习效率和编辑准确性。通过这些策略，我们在VIBE基准上显著提升了单任务和多任务涂鸦引导编辑的性能，取得了最先进的结果。我们将公开发布我们的数据集和模型。
英文摘要：
Scribble-guided image editing allows users to combine simple scribble annotations with text prompts to specify both where and how an image should be edited, enabling flexible interaction with precise spatial control. However, existing models still exhibit unstable performance under this paradigm, especially in multi-task scenarios. To improve performance, we conduct empirical studies using an open-source editing model and reveal an asymmetry in generalization: instruction-level generalization, including across editing tasks and from single-task to multi-task settings, is more challenging than image-domain generalization, such as from synthetic to real-world images or from mosaicked to regular images. This suggests that the primary bottleneck lies in insufficient learning for diverse editing instructions rather than in the image domain gap. Motivated by this insight, we propose three strategies: (a) a Coverage-then-Realism Curriculum, a two-stage pipeline that first builds large-scale synthetic, instruction-rich data for broad task supervision, then curates a small set of real-world data to refine generation realism; (b) Multi-Task Mosaicking, which constructs multi-task training samples by concatenating single-task examples at nearly zero cost while enabling the learned capability to generalize to non-mosaicked images; and (c) an Edit-Focused Loss, which leverages the changed regions between input and output images in synthetic data to focus training on edited regions, improving both learning efficiency and editing accuracy. With these strategies, we substantially improve both single-task and multi-task scribble-guided editing on the VIBE benchmark, achieving state-of-the-art results. We will publicly release our dataset and model.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
