VENUS: 使用场景图进行视觉编辑的噪声反演
VENUS: Visual Editing with Noise Inversion Using Scene Graphs
AI总结：
VENUS是一种无需训练的场景图引导图像编辑框架，通过噪声反演和拆分提示策略提升图像编辑的保真度和语义一致性。
摘要：
最先进的基于文本的图像编辑模型往往难以在背景保留与语义一致性之间取得平衡，通常导致要么生成完全新的图像，要么输出无法实现预期编辑。相比之下，基于场景图的图像编辑通过提供语义实体及其关系的结构化表示，从而提供了改进的可控性。然而，现有的场景图编辑方法通常依赖于模型微调，这会带来高昂的计算成本并限制可扩展性。为此，我们引入了VENUS（使用场景图进行视觉编辑的噪声反演），一种无需训练的场景图引导图像编辑框架。具体而言，VENUS采用了拆分提示条件策略，将编辑目标对象与其背景上下文分离，同时利用噪声反演在未编辑区域保持保真度。此外，我们提出的方法整合了从多模态大语言模型中提取的场景图与扩散骨干网络，无需任何额外训练。从经验上，VENUS在PIE-Bench上显著提高了背景保留和语义对齐，将PSNR从22.45提高到24.80，SSIM从0.79提高到0.84，并将LPIPS从0.100降低到0.070，相对于最先进的场景图编辑模型（SGEdit）。此外，VENUS通过CLIP相似性（24.97 vs. 24.19）增强了语义一致性。在EditVal上，VENUS实现了最高保真度，得分为0.87 DINO分数，并且关键的是，将每张图像的运行时间从6-10分钟减少到仅20-30秒。除了基于场景图的编辑外，VENUS还超越了强大的基于文本的编辑基线，如LEdit++和P2P+DirInv，从而在两种范式中都展示了持续的改进。
英文摘要：
State-of-the-art text-based image editing models often struggle to balance background preservation with semantic consistency, frequently resulting either in the synthesis of entirely new images or in outputs that fail to realize the intended edits. In contrast, scene graph-based image editing addresses this limitation by providing a structured representation of semantic entities and their relations, thereby offering improved controllability. However, existing scene graph editing methods typically depend on model fine-tuning, which incurs high computational cost and limits scalability. To this end, we introduce VENUS (Visual Editing with Noise inversion Using Scene graphs), a training-free framework for scene graph-guided image editing. Specifically, VENUS employs a split prompt conditioning strategy that disentangles the target object of the edit from its background context, while simultaneously leveraging noise inversion to preserve fidelity in unedited regions. Moreover, our proposed approach integrates scene graphs extracted from multimodal large language models with diffusion backbones, without requiring any additional training. Empirically, VENUS substantially improves both background preservation and semantic alignment on PIE-Bench, increasing PSNR from 22.45 to 24.80, SSIM from 0.79 to 0.84, and reducing LPIPS from 0.100 to 0.070 relative to the state-of-the-art scene graph editing model (SGEdit). In addition, VENUS enhances semantic consistency as measured by CLIP similarity (24.97 vs. 24.19). On EditVal, VENUS achieves the highest fidelity with a 0.87 DINO score and, crucially, reduces per-image runtime from 6-10 minutes to only 20-30 seconds. Beyond scene graph-based editing, VENUS also surpasses strong text-based editing baselines such as LEDIT++ and P2P+DirInv, thereby demonstrating consistent improvements across both paradigms.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
