注意你的步骤：基于文本指令的局部图像与场景编辑
Watch Your Steps: Local Image and Scene Editing by Text Instructions
AI总结：
本文提出利用InstructPix2Pix预测差异生成相关性图来定位文本指令对应的编辑区域，并扩展为3D相关性场，实现图像与NeRF场景的精准局部编辑。
摘要：
去噪扩散模型已经实现了高质量的图像生成和编辑。我们提出了一种方法，用于定位文本指令中隐含的期望编辑区域。我们利用InstructPix2Pix（IP2P），并识别在有指令和无指令条件下IP2P预测之间的差异。该差异被称为相关性图。相关性图传达了改变每个像素以实现编辑的重要性，并被用于引导修改。这种引导确保不相关的像素保持不变。相关性图还被进一步用于以神经辐射场形式增强3D场景的文本引导编辑质量。在训练视图的相关性图上训练一个场，称为相关性场，定义应在其中进行修改的3D区域。我们根据从相关性场渲染出的相关性图对训练视图进行迭代更新。我们的方法在图像和NeRF编辑任务上均达到了最先进的性能。项目页面：https://ashmrz.github.io/WatchYourSteps/
英文摘要：
Denoising diffusion models have enabled high-quality image generation and editing. We present a method to localize the desired edit region implicit in a text instruction. We leverage InstructPix2Pix (IP2P) and identify the discrepancy between IP2P predictions with and without the instruction. This discrepancy is referred to as the relevance map. The relevance map conveys the importance of changing each pixel to achieve the edits, and is used to to guide the modifications. This guidance ensures that the irrelevant pixels remain unchanged. Relevance maps are further used to enhance the quality of text-guided editing of 3D scenes in the form of neural radiance fields. A field is trained on relevance maps of training views, denoted as the relevance field, defining the 3D region within which modifications should be made. We perform iterative updates on the training views guided by rendered relevance maps from the relevance field. Our method achieves state-of-the-art performance on both image and NeRF editing tasks. Project page: https://ashmrz.github.io/WatchYourSteps/
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
