超高分辨率图像修复:基于补丁内容一致性适配器
Ultra High-Resolution Image Inpainting with Patch-Based Content Consistency Adapter
AI总结:
本文提出Patch-Adapter两阶段适配器框架,通过双上下文与参考补丁机制将扩散模型分辨率扩展至4K+,在OpenImages等数据集上实现高分辨率文本引导图像修复的最优性能。
摘要:
本文提出了Patch-Adapter,一个用于高分辨率文本引导图像修复的有效框架。与局限于较低分辨率的现有方法不同,我们的方法实现了4K+分辨率,同时保持了精确的内容一致性和提示对齐,这两个是图像修复中随着分辨率和纹理复杂度增加而加剧的关键挑战。Patch-Adapter采用两阶段适配器架构,将扩散模型的分辨率从1K扩展到4K+,而无需进行结构性改造:(1)双上下文适配器在降低的分辨率下学习掩蔽区域与未掩蔽区域之间的连贯性,以建立全局结构一致性;(2)参考补丁适配器实现补丁级注意力机制,用于全分辨率修复,通过自适应特征融合保持局部细节保真度。这种双阶段架构通过将全局语义与局部细化解耦,独特地解决了高分辨率修复中的可扩展性差距。实验表明,Patch-Adapter不仅解决了大规模修复中常见的伪影问题,而且在OpenImages和Photo-Concept-Bucket数据集上达到了最先进的性能,在感知质量和文本提示遵循方面均优于现有方法。
英文摘要:
In this work, we present Patch-Adapter, an effective framework for high-resolution text-guided image inpainting. Unlike existing methods limited to lower resolutions, our approach achieves 4K+ resolution while maintaining precise content consistency and prompt alignment, two critical challenges in image inpainting that intensify with increasing resolution and texture complexity. Patch-Adapter leverages a two-stage adapter architecture to scale the diffusion model's resolution from 1K to 4K+ without requiring structural overhauls: (1) Dual Context Adapter learns coherence between masked and unmasked regions at reduced resolutions to establish global structural consistency; and (2) Reference Patch Adapter implements a patch-level attention mechanism for full-resolution inpainting, preserving local detail fidelity through adaptive feature fusion. This dual-stage architecture uniquely addresses the scalability gap in high-resolution inpainting by decoupling global semantics from localized refinement. Experiments demonstrate that Patch-Adapter not only resolves artifacts common in large-scale inpainting but also achieves state-of-the-art performance on the OpenImages and Photo-Concept-Bucket datasets, outperforming existing methods in both perceptual quality and text-prompt adherence.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。