InstEditSeg：用于息肉与皮肤病变分割的指令驱动图像编辑方法
InstEditSeg: Instruction-Driven Image Editing for Polyp and Skin Lesion Segmentation
- School of Information and Control Engineering, Southwest University of Science and Technology（西南科技大学信息与控制工程学院）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
本研究提出InstEditSeg生成框架，将医学分割转化为指令驱动的图像编辑问题，通过引入DINOv3等技术优化，在息肉与皮肤病变分割任务中实现了优异性能及跨域泛化等优势。
AI中文摘要：
息肉和皮肤病变的精准分割对临床诊断至关重要，但现有方法难以应对低对比度、模糊边界及跨域分布差异问题。判别网络与多数基于扩散的分割方法仅输出独立的二值掩码，未能充分利用大规模预训练生成模型的视觉先验。我们提出InstEditSeg，这是一个统一的生成框架，将医学分割重新表述为指令驱动的图像编辑问题。该模型不生成掩码，而是基于文本指令在原始图像上渲染彩色编码的叠加层，使编辑后的输出与潜扩散模型学习到的自然图像分布对齐，同时缩小自然图像与医学图像之间的域差距。为恢复精细解剖结构，我们引入DINOv3作为辅助视觉编码器，并设计DINO特征引导块构建多尺度特征金字塔；该金字塔通过通道拼接和零初始化卷积融合到扩散U-Net中，可注入分层判别先验而不干扰预训练权重。我们还采用双分支无分类器引导策略，每个去噪步骤仅需两次前向传播，降低了推理成本。在息肉与皮肤病变基准测试中，该框架的准确率可与强大的判别基线相媲美，且展现出生成式表述的具体优势：对未见过的数据具有更优的跨域泛化能力、更完整的多病变分割、指令条件下的任务控制及采样灵活性。我们还分析了该范式的优势与局限，包括对颜色敏感及不支持属性条件选择。代码可在以下网址获取：this https URL
英文摘要：
Accurate segmentation of polyps and skin lesions is pivotal for clinical diagnosis, yet existing methods struggle with low contrast, ambiguous boundaries, and cross-domain distribution discrepancies. Discriminative networks and most diffusion-based segmentation approaches predict standalone binary masks, leaving the visual priors of large-scale pretrained generative models largely unexploited. We propose InstEditSeg, a unified generative framework that reformulates medical segmentation as an instruction-driven image editing problem. Instead of emitting a mask, the model renders a color-coded overlay on the original image, conditioned on a textual instruction, so that the edited output aligns with the natural image distribution learned by latent diffusion models and mitigates the domain gap between natural and medical imagery. To recover fine anatomical structures, we introduce DINOv3 as an auxiliary visual encoder and a DINO Feature Guidance Block that builds a multi-scale feature pyramid. The pyramid is fused into the diffusion U-Net by channel concatenation and zero-initialized convolution so that hierarchical discriminative priors can be injected without perturbing the pretrained weights. A dual-branch classifier-free guidance strategy requiring only two forward passes per denoising step reduces inference cost. On polyp and skin lesion benchmarks the framework achieves accuracy competitive with strong discriminative baselines, and it further demonstrates concrete advantages of the generative formulation: notably better cross-domain generalization on unseen data, more complete multi-lesion segmentation, instruction-conditioned task control, and sampling flexibility. We also analyze the strengths and limitations of the paradigm, including its color sensitivity and unsupported attribute-conditioned selection. Code is available at: https://github.com/wincharm001/InstEditSeg.
