基于视觉自回归模型的掩码logit调整的提示引导图像编辑
Prompt-Guided Image Editing with Masked Logit Nudging in Visual Autoregressive Models
AI总结：
本文提出掩码logit调整方法，通过调整模型预测logits实现提示引导的图像编辑，保留无关区域，提升编辑质量和重建效果，优于现有方法。
摘要：
我们解决了视觉自回归模型中提示引导图像编辑的问题。给定源图像和目标文本提示，我们旨在根据目标提示修改源图像，同时保留所有与请求编辑无关的区域。为此，我们提出了掩码logit调整，利用源图像标记图引入指导步骤，使模型在目标提示下的预测与这些源标记图对齐。具体而言，我们将固定的源编码转换为logits，通过源-目标提示定义的语义轨迹调整模型预测logits。编辑仅在通过专用掩码方案获得的空间掩码内进行。然后，我们引入了细化步骤以纠正量化误差并提高重建质量。我们的方法在PIE基准测试中实现了512px和1024px分辨率的最佳图像编辑性能。除了编辑外，我们的方法还实现了忠实的重建，并在COCO和OpenImages上优于先前方法。总体而言，我们的方法优于VAR相关方法，与扩散模型相比具有可比或更优的性能，同时速度更快。代码可在'https://github.com/AmirMaEl/MLN'获得。
英文摘要：
We address the problem of prompt-guided image editing in visual autoregressive models. Given a source image and a target text prompt, we aim to modify the source image according to the target prompt, while preserving all regions which are unrelated to the requested edit. To this end, we present Masked Logit Nudging, which uses the source image token maps to introduce a guidance step that aligns the model's predictions under the target prompt with these source token maps. Specifically, we convert the fixed source encodings into logits using the VAR encoding, nudging the model's predicted logits towards the targets along a semantic trajectory defined by the source-target prompts. Edits are applied only within spatial masks obtained through a dedicated masking scheme that leverages cross-attention differences between the source and edited prompts. Then, we introduce a refinement to correct quantization errors and improve reconstruction quality. Our approach achieves the best image editing performance on the PIE benchmark at 512px and 1024px resolutions. Beyond editing, our method delivers faithful reconstructions and outperforms previous methods on COCO at 512px and OpenImages at 1024px. Overall, our method outperforms VAR-related approaches and achieves comparable or even better performance than diffusion models, while being much faster. Code is available at 'https://github.com/AmirMaEl/MLN'.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
