PANDORA:基于像素注意力溶解和潜在引导的零样本物体移除
PANDORA: Pixel-wise Attention Dissolution and Latent Guidance for Zero-Shot Object Removal
AI总结:
本文提出PANDORA框架,通过像素注意力溶解和局部注意力解缠引导,实现无需提示的多物体移除,具有高精度和可扩展性。
摘要:
从自然图像中移除物体具有挑战性,因为合成语义连贯内容的同时保持背景完整性困难。现有方法常依赖微调、提示工程或推理时优化,但仍存在纹理不一致、刚性伪影、前景-背景解缠不足和多物体移除扩展性差等问题。本文提出PANDORA框架,直接在预训练的文本到图像扩散模型上操作,无需微调、提示或优化。我们提出像素注意力溶解,通过消除掩码像素最相关的注意力键来移除物体,有效消除物体自注意力流,使背景上下文主导重建。我们进一步引入局部注意力解缠引导,引导去噪过程向有利于干净物体移除的潜在流方向发展。这些组件共同实现单次通过的精确、非刚性、无提示和可扩展的多物体擦除。实验表明,与现有最先进方法相比,具有更优的视觉保真度和语义合理性。项目页面可在https://vdkhoi20.github.io/PANDORA访问。
英文摘要:
Removing objects from natural images is challenging due to difficulty of synthesizing semantically coherent content while preserving background integrity. Existing methods often rely on fine-tuning, prompt engineering, or inference-time optimization, yet still suffer from texture inconsistency, rigid artifacts, weak foreground-background disentanglement, and poor scalability for multi-object removal. We propose a novel zero-shot object removal framework, namely PANDORA, that operates directly on pre-trained text-to-image diffusion models, requiring no fine-tuning, prompts, or optimization. We propose Pixel-wise Attention Dissolution to remove object by nullifying the most correlated attention keys for masked pixels, effectively eliminating the object from self-attention flow and allowing background context to dominate reconstruction. We further introduce Localized Attentional Disentanglement Guidance to steer denoising toward latent manifolds favorable to clean object removal. Together, these components enable precise, non-rigid, prompt-free, and scalable multi-object erasure in a single pass. Experiments demonstrate superior visual fidelity and semantic plausibility compared to state-of-the-art methods. The project page is available at https://vdkhoi20.github.io/PANDORA.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。