基于场景图理解的复杂场景图像编辑
Complex Scene Image Editing by Scene Graph Comprehension
AI总结：
本文提出SGC-Net两阶段方法，利用场景图理解预测目标物体RoI并结合条件扩散模型实现复杂场景图像编辑，在CLEVR和Visual Genome上显著优于先前工作。
摘要：
条件扩散模型已在文本引导的语义图像编辑等多种任务中展现出卓越性能。此前的工作需要人类用户手动识别图像区域，或使用仅在以物体为中心的操作中表现良好的物体检测器。例如，若输入图像包含多个具有相同语义的物体（如一群鸟），物体检测器可能难以识别和定位目标物体，更不用说准确地对其进行操作。为解决这些挑战，我们提出了一种两阶段方法，通过场景图理解（SGC-Net）实现复杂场景图像编辑。在第一阶段，我们训练一个感兴趣区域（RoI）预测网络，该网络使用场景图并预测目标物体的位置。与仅基于物体类别的物体检测方法不同，我们的方法能够通过理解复杂场景中的物体及其语义关系来准确识别目标物体。第二阶段使用条件扩散模型基于我们的RoI预测来编辑图像。我们在CLEVR和Visual Genome数据集上评估了所提方法的有效性。我们在CLEVR上报告了SSIM提升8个点，并且在Visual Genome上，我们编辑的图像被人类用户偏好的比例比先前工作高出9-33%，验证了所提方法的有效性。代码可在github.com/Zhongping-Zhang/SGC_Net获取。
英文摘要：
Conditional diffusion models have demonstrated impressive performance on various tasks like text-guided semantic image editing. Prior work requires image regions to be identified manually by human users or use an object detector that only perform well for object-centric manipulations. For example, if an input image contains multiple objects with the same semantic meaning (such as a group of birds), object detectors may struggle to recognize and localize the target object, let alone accurately manipulate it. To address these challenges, we propose a two-stage method for achieving complex scene image editing by Scene Graph Comprehension (SGC-Net). In the first stage, we train a Region of Interest (RoI) prediction network that uses scene graphs and predict the locations of the target objects. Unlike object detection methods based solely on object category, our method can accurately recognize the target object by comprehending the objects and their semantic relationships within a complex scene. The second stage uses a conditional diffusion model to edit the image based on our RoI predictions. We evaluate the effectiveness of our approach on the CLEVR and Visual Genome datasets. We report an 8 point improvement in SSIM on CLEVR and our edited images were preferred by human users by 9-33% over prior work on Visual Genome, validating the effectiveness of our proposed method. Code is available at github.com/Zhongping-Zhang/SGC_Net.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
