ObjectDrop：为逼真物体移除与插入引导反事实
ObjectDrop: Bootstrapping Counterfactuals for Photorealistic Object Removal and Insertion
AI总结：
针对扩散模型在图像编辑中违反物理定律的问题，提出基于反事实数据集的ObjectDrop方法，通过微调扩散模型实现逼真的物体移除与插入，并利用引导监督扩展数据集以显著提升对场景影响的建模效果。
摘要：
扩散模型彻底改变了图像编辑，但生成的图像常常违反物理定律，特别是物体对场景的影响，例如遮挡、阴影和反射。通过分析自监督方法的局限性，我们提出了一种以“反事实”数据集为核心的实用解决方案。我们的方法包括在移除单个物体前后捕获场景，同时尽量减少其他变化。通过在该数据集上微调扩散模型，我们不仅能移除物体，还能移除其对场景的影响。然而，我们发现将此方法应用于逼真物体插入需要不切实际的大型数据集。为应对这一挑战，我们提出了引导监督；利用在小规模反事实数据集上训练的物体移除模型，我们在合成上大幅扩展了该数据集。我们的方法在逼真物体移除和插入方面显著优于先前的方法，特别是在建模物体对场景的影响方面。
英文摘要：
Diffusion models have revolutionized image editing but often generate images that violate physical laws, particularly the effects of objects on the scene, e.g., occlusions, shadows, and reflections. By analyzing the limitations of self-supervised approaches, we propose a practical solution centered on a \q{counterfactual} dataset. Our method involves capturing a scene before and after removing a single object, while minimizing other changes. By fine-tuning a diffusion model on this dataset, we are able to not only remove objects but also their effects on the scene. However, we find that applying this approach for photorealistic object insertion requires an impractically large dataset. To tackle this challenge, we propose bootstrap supervision; leveraging our object removal model trained on a small counterfactual dataset, we synthetically expand this dataset considerably. Our approach significantly outperforms prior methods in photorealistic object removal and insertion, particularly at modeling the effects of objects on the scene.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
