SketchSense：学习解释用于图像修复的不完美草图引导
SketchSense: Learning to Interpret Imperfect Sketch Guidance for Image Inpainting
浏览论文内容
中文总结 AI 辅助
SketchSense是同步去噪RGB与结构流的框架，通过双向注意力融合等技术解释不完美草图引导，在图像修复的质量和结构保真度上较现有方法有显著提升。
中文摘要 AI 辅助
草图引导的图像修复提供了直观的结构控制，但真实草图通常混合了可靠的全局意图与局部拥挤、错位、不完整或刻意非常规的笔触。现有方法通常要么在整个去噪过程中保留输入草图作为固定条件，要么在RGB合成前将其细化为干净结构。前者假设笔触均匀可靠，会将局部错误传播到整个生成过程；后者必须在外观和语义上下文可用前解决模糊结构。我们提出SketchSense，这一框架通过同步去噪交互的RGB和结构流来解释不完美的草图引导。双向注意力融合将外观生成与结构恢复相结合，生成细化结构以展现模型不断演变的草图解释。短语级目标使两个流的语义基础对齐。草图感知空间调节通过调制注意力和融合过程，使草图使用适应当地生成状态，而可选的带符号先验将“保留-修正”意图注入特征表示和注意力行为。在自然和结构复杂图像上的实验显示，与现有方法相比，在修复质量和结构保真度上均有显著提升。
英文摘要
Sketch-guided image inpainting provides intuitive structural control, yet real sketches often mix reliable global intent with locally crowded, displaced, incomplete, or deliberately unconventional strokes. Existing approaches typically either retain the input sketch as a fixed condition throughout denoising or refine it into a clean structure before RGB synthesis. The former assumes uniformly reliable strokes and can propagate local errors throughout generation; the latter must resolve ambiguous structure before emerging appearance and semantic context become available. We propose SketchSense, a framework that interprets imperfect sketch guidance by synchronously denoising interacting RGB and structure streams. Bidirectional Attention Fusion couples appearance generation with structural recovery, producing a refined structure that exposes the model's evolving sketch interpretation. A phrase-level objective aligns the semantic grounding of the two streams. Sketch-Aware Spatial Regulation further adapts sketch use to local generation states by modulating attention and the fusion process, while an optional signed prior injects preserve-versus-correct intent into feature representations and attention behavior. Experiments on natural and structurally complex imagery show substantial gains over existing methods in both restoration quality and structural fidelity.
发表机构
- Fudan University（复旦大学）
机构由 AI 辅助整理，请以论文原文为准。
