基于视觉自回归模型的免训练文本引导图像编辑
Training-Free Text-Guided Image Editing with Visual Autoregressive Model
AI总结：
提出基于VAR的免训练文本引导图像编辑框架，通过缓存机制和自适应掩码策略实现精确局部编辑，无需显式反演，在速度与质量上超越现有扩散及修正流方法。
摘要：
文本引导的图像编辑是一项重要任务，允许用户通过自然语言描述修改图像。近期扩散模型和修正流的进展显著提升了编辑质量，主要依赖反演技术从输入图像中提取结构化噪声。然而，反演的不准确会传播误差，导致非预期修改并损害保真度。此外，即使反演完美，文本提示与图像特征之间的纠缠也常导致仅意图局部编辑时发生全局变化。为应对这些挑战，我们提出一种基于VAR（Visual AutoRegressive modeling，视觉自回归建模）的新型文本引导图像编辑框架，该框架消除了显式反演的需求，同时确保精确可控的修改。我们的方法引入一种缓存机制，存储原始图像的token索引和概率分布，捕捉源提示与图像之间的关系。利用此缓存，我们设计了一种自适应细粒度掩码策略，动态识别并约束对相关区域的修改，防止非预期变化。一种token重组方法进一步优化编辑过程，增强多样性、保真度和控制力。我们的框架以免训练方式运行，实现高保真编辑且推理速度更快，处理1K分辨率图像最快仅需1.2秒。大量实验表明，我们的方法在定量指标和视觉质量上均达到与现有基于扩散和修正流的方法相当甚至更优的性能。代码将公开发布。
英文摘要：
Text-guided image editing is an essential task that enables users to modify images through natural language descriptions. Recent advances in diffusion models and rectified flows have significantly improved editing quality, primarily relying on inversion techniques to extract structured noise from input images. However, inaccuracies in inversion can propagate errors, leading to unintended modifications and compromising fidelity. Moreover, even with perfect inversion, the entanglement between textual prompts and image features often results in global changes when only local edits are intended. To address these challenges, we propose a novel text-guided image editing framework based on VAR (Visual AutoRegressive modeling), which eliminates the need for explicit inversion while ensuring precise and controlled modifications. Our method introduces a caching mechanism that stores token indices and probability distributions from the original image, capturing the relationship between the source prompt and the image. Using this cache, we design an adaptive fine-grained masking strategy that dynamically identifies and constrains modifications to relevant regions, preventing unintended changes. A token reassembling approach further refines the editing process, enhancing diversity, fidelity, and control. Our framework operates in a training-free manner and achieves high-fidelity editing with faster inference speeds, processing a 1K resolution image in as fast as 1.2 seconds. Extensive experiments demonstrate that our method achieves performance comparable to, or even surpassing, existing diffusion- and rectified flow-based approaches in both quantitative metrics and visual quality. The code will be released.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
