用于指导图像编辑的视觉自回归建模
Visual Autoregressive Modeling for Instruction-Guided Image Editing
AI总结：
VAREdit通过自回归框架实现高效精准的图像编辑，优于扩散模型并在速度和质量上取得显著提升。
摘要：
近年来，扩散模型在指导图像编辑中的视觉保真度方面取得了显著进展。然而，其全局去噪过程本质上将编辑区域与整个图像上下文纠缠在一起，导致意外的虚假修改和对编辑指令的遵守程度降低。相比之下，自回归模型提供了一种不同的范式，通过将图像合成作为离散视觉标记的序列过程来建模。其因果性和组合性机制自然地绕过了扩散方法的遵守挑战。在本文中，我们提出了VAREdit，一种视觉自回归（VAR）框架，将图像编辑重新表述为下一尺度预测问题。在源图像特征和文本指令的条件下，VAREdit生成多尺度目标特征以实现精确编辑。该范式的核心挑战是如何有效地对源图像标记进行条件化。我们发现，最细粒度的源特征无法有效指导更粗粒度的目标特征的预测。为弥合这一差距，我们引入了尺度对齐参考（SAR）模块，该模块将尺度匹配的条件信息注入到第一个自注意力层中。VAREdit在编辑遵守和效率方面均取得了显著进步。在EMU-Edit和PIE-Bench基准上，VAREdit在CLIP和GPT分数方面均显著优于领先扩散方法。此外，VAREdit在1.2秒内完成512×512的编辑，比同样大小的UltraEdit快2.2倍。代码可在：https://github.com/HiDream-ai/VAREdit。
英文摘要：
Recent advances in diffusion models have brought remarkable visual fidelity to instruction-guided image editing. However, their global denoising process inherently entangles the edited region with the entire image context, leading to unintended spurious modifications and compromised adherence to editing instructions. In contrast, autoregressive models offer a distinct paradigm by formulating image synthesis as a sequential process over discrete visual tokens. Their causal and compositional mechanism naturally circumvents the adherence challenges of diffusion-based methods. In this paper, we present VAREdit, a visual autoregressive (VAR) framework that reframes image editing as a next-scale prediction problem. Conditioned on source image features and text instructions, VAREdit generates multi-scale target features to achieve precise edits. A core challenge in this paradigm is how to effectively condition the source image tokens. We observe that finest-scale source features cannot effectively guide the prediction of coarser target features. To bridge this gap, we introduce a Scale-Aligned Reference (SAR) module, which injects scale-matched conditioning information into the first self-attention layer. VAREdit demonstrates significant advancements in both editing adherence and efficiency. On EMU-Edit and PIE-Bench benchmarks, VAREdit outperforms leading diffusion-based methods by a substantial margin in terms of both CLIP and GPT scores. Moreover, VAREdit completes a 512$\times$512 editing in 1.2 seconds, making it 2.2$\times$ faster than the similarly sized UltraEdit. Code is available at: https://github.com/HiDream-ai/VAREdit.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
