SketchAssist:一种用于语义编辑和精确局部重绘的实用助手
SketchAssist: A Practical Assistant for Semantic Edits and Precise Local Redrawing
AI总结:
提出SketchAssist,一种结合指令引导编辑和线条引导区域重绘的交互式草图助手,通过可控数据生成管道和基于DiT的统一框架(集成任务引导的混合专家模块)实现高效、可控的草图操作,在语义和结构一致性上达到最先进性能。
摘要:
草图编辑需要同时处理高级语义变化和精确的局部重绘,这种组合对于稀疏且风格敏感的线条艺术尤其具有挑战性。与自然图像不同,草图依赖于最小的视觉线索,使得现有方法难以在保持整体一致性的同时协调全局语义修改与细粒度结构控制。我们提出了SketchAssist,一种交互式草图助手,它统一了指令引导编辑和线条引导区域重绘,在保持整体构图的同时实现高效且可控的草图操作。为了支持这一任务,我们引入了一个可控数据生成管道,该管道构建具有精确属性变化的结构化编辑序列,并在多步修改中保持结构对齐,同时通过保持风格的变换扩展风格多样性。基于这些数据,SketchAssist采用基于DiT的统一框架,使用多通道输入表示在单一接口内编码草图、掩码和引导信号。为了进一步处理不同的编辑模式,我们将任务引导的混合专家(T-MoE)集成到LoRA层中,实现对语义和结构引导的自适应控制。大量实验表明,在两个任务上都达到了最先进的性能,与最近的方法相比,实现了更强的指令遵循以及改进的结构和风格一致性。总之,我们的方法为草图编辑提供了一种实用且可控的解决方案。
英文摘要:
Sketch editing requires jointly handling high-level semantic changes and precise local redrawing, a combination that is particularly challenging for sparse, style-sensitive line art. Unlike natural images, sketches rely on minimal visual cues, making it difficult for existing methods to reconcile global semantic modifications with fine-grained structural control while preserving overall coherence. We present SketchAssist, an interactive sketch assistant that unifies instruction-guided editing with line-guided region redrawing, enabling efficient and controllable sketch manipulation while preserving overall composition. To support this task, we introduce a controllable data generation pipeline that constructs structured edit sequences with precise attribute variations and maintains structural alignment across multi-step modifications, while expanding stylistic diversity via style-preserving transformations. Building on this data, SketchAssist adopts a unified framework based on DiT, using a multi-channel input representation to encode sketches, masks, and guidance signals within a single interface. To further handle different editing modes, we integrate a Task-guided Mixture-of-Experts (T-MoE) into LoRA layers, enabling adaptive control over semantic and structural guidance. Extensive experiments demonstrate state-of-the-art performance on both tasks, achieving strong instruction adherence and improved structural and style consistency compared to recent methods. Together, our method provide a practical and controllable solution for sketch editing.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。