通过逐步对齐实现图像编辑的文本与视觉提示融合
Textual and Visual Prompt Fusion for Image Editing via Step-Wise Alignment
AI总结：
针对图像编辑中视觉参考语义控制不足与文本引导视觉质量受损的问题，本文提出将视觉参考和文本提示融合进冻结扩散模型语义潜空间的框架，以小型网络实现高质量、逼真且可控的编辑。
摘要：
去噪扩散模型在图像编辑领域的应用正变得越来越流行。然而，当前方法通常依赖于图像引导方法或文本引导方法：前者提供视觉参考，但缺乏对语义一致性的控制；后者确保与文本引导对齐，却会牺牲视觉质量。为解决这一问题，我们提出一个框架，将生成的视觉参考与文本引导的融合集成到一个冻结的预训练扩散模型的语义潜空间中。仅使用一个极小的神经网络，该框架就能由文本提示直观驱动，对多样内容和属性提供控制。与最先进方法相比，该框架在多个基准数据集上生成更高质量的图像，同时提供逼真的编辑效果。
英文摘要：
The use of denoising diffusion models is becoming increasingly popular in the field of image editing. However, current approaches often rely on either image-guided methods, which provide a visual reference but lack control over semantic consistency, or text-guided methods, which ensure alignment with the text guidance but compromise visual quality. To resolve this issue, we propose a framework that integrates a fusion of generated visual references and text guidance into the semantic latent space of a \textit{frozen} pre-trained diffusion model. Using only a tiny neural network, our framework provides control over diverse content and attributes, driven intuitively by the text prompt. Compared to state-of-the-art methods, the framework generates images of higher quality while providing realistic editing effects across various benchmark datasets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
