发表机构
Institute of Cognitive Science, Osnabrück University（奥斯纳布吕克大学认知科学研究所）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
提出零样本物体移除框架，结合SAM掩膜、BLIP字幕、DDIM反演及注意力掩蔽等，实现无需训练的上下文一致修复，并通过消融验证有效性。
AI 中文摘要
从真实图像中移除一个物体，不仅需要在掩膜内合成看似合理的内容：该方法必须抑制残留的物体特征，保持未编辑场景的完整性，并生成与周围背景一致的替换内容。本文从分阶段的角度处理物体移除问题，提出了一种基于冻结的预训练稳定扩散模型的零样本框架，用于受限潜在空间修复，无需任务特定训练或模型微调。该方法将基于SAM的掩膜构建、BLIP图像字幕条件化、DDIM反演、背景加权掩膜空文本优化、解码器自注意力掩蔽、掩膜外硬性潜在锚定以及局部重噪声-去噪细化集成到一个统一流程中。通过定性示例、定量局部一致性指标和消融研究对该方法进行了评估。结果表明，该方法能有效移除物体并生成上下文一致的替换内容。消融研究表明，背景加权掩膜NTI对结构复杂的背景特别有益，而在其他评估示例中，无NTI变体已足够。重复细化进一步减少了初次编辑后残留的物体痕迹和边界伪影。
英文摘要
Removing an object from a real image requires more than synthesizing plausible content within a mask: the method must suppress residual object features, preserve the unedited scene, and generate replacement content that is consistent with the surrounding background. This paper approaches object removal from a stage-based perspective and proposes a zero-shot framework for constrained latent inpainting with a frozen pretrained Stable Diffusion model, requiring no task-specific training or model fine-tuning. The method integrates SAM-based mask construction, BLIP image-caption conditioning, DDIM inversion, background-weighted masked null-text optimization, decoder self-attention masking, hard outside-mask latent anchoring, and localized renoise--denoise refinement into a unified pipeline. The method is evaluated through qualitative examples, quantitative local-consistency metrics, and ablation studies. The results demonstrate effective object removal and context-consistent replacement content. The ablations indicate that background-weighted masked NTI is particularly beneficial for structurally complex backgrounds, whereas the no-NTI variant is sufficient in other evaluated examples. Repeated refinement further reduces object remnants and boundary artifacts remaining after the primary editing pass.
CommentsCode available at https://github.com/arman-taghizadeh/zero-shot-diffusion-object-removal
