arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2412.15541cs.CVcs.AI
收藏

ChangeDiff:基于扩散模型且具有灵活文本提示的多时相变化检测数据生成器

ChangeDiff: A Multi-Temporal Change Detection Data Generator with Flexible Text Prompts via Diffusion Model

Xidian University(西安电子科技大学);Hefei University of Technology(合肥工业大学)

Qi Zang, Jiayi Yang, Shuang Wang, Dong Zhao, Wenjun Yi, Zhun Zhong

AI总结:

本文提出多时相语义变化检测数据生成器ChangeDiff,通过文本到布局和布局到图像两步生成变化数据,并设计多类分布引导的文本提示及类分布细化损失,提升了变化检测器的准确性与可迁移性。

摘要:

在像素级标注的支持下,数据驱动的深度学习模型在变化检测(CD)领域取得了巨大进展。然而,收集多样化数据并进行人工标注成本高昂、耗时费力且需要大量专业知识。现有的用于CD数据合成的生成方法在解决这一问题上展现出竞争潜力,但仍面临以下局限性:1)难以灵活控制变化事件,2)依赖额外数据来训练数据生成器,3)仅关注特定的变化检测任务。为此,本文聚焦于语义变化检测(SCD)任务,通过探索强大的扩散模型,开发了一种多时相SCD数据生成器ChangeDiff。ChangeDiff创新性地分两步生成变化数据:首先,它利用文本提示和文本到布局(T2L)模型创建连续布局,然后采用布局到图像(L2I)将这些布局转换为图像。具体而言,我们提出了多类分布引导的文本提示(MCDG-TP),允许通过可控的类别及其对应比例灵活生成布局。随后,为了将T2L模型泛化到所提出的MCDG-TP,进一步设计了类分布细化损失作为训练监督。为了将文本到图像扩散模型泛化到所提出的MCDG-TP,设计了类分布细化损失作为训练监督。对于后者,提出了三种模式下的MCDG-TP,用于从各种文本中合成新的布局掩码。我们生成的数据在时间连续性、空间多样性和质量真实感方面展现出显著进步,赋予了变化检测器更高的准确性和可迁移性。代码可在https://github.com/DZhaoXd/ChangeDiff获取。

英文摘要:

Data-driven deep learning models have enabled tremendous progress in change detection (CD) with the support of pixel-level annotations. However, collecting diverse data and manually annotating them is costly, laborious, and knowledge-intensive. Existing generative methods for CD data synthesis show competitive potential in addressing this issue but still face the following limitations: 1) difficulty in flexibly controlling change events, 2) dependence on additional data to train the data generators, 3) focus on specific change detection tasks. To this end, this paper focuses on the semantic CD (SCD) task and develops a multi-temporal SCD data generator ChangeDiff by exploring powerful diffusion models. ChangeDiff innovatively generates change data in two steps: first, it uses text prompts and a text-to-layout (T2L) model to create continuous layouts, and then it employs layout-to-image (L2I) to convert these layouts into images. Specifically, we propose multi-class distribution-guided text prompts (MCDG-TP), allowing for layouts to be generated flexibly through controllable classes and their corresponding ratios. Subsequently, to generalize the T2L model to the proposed MCDG-TP, a class distribution refinement loss is further designed as training supervision. %For the former, a multi-classdistribution-guided text prompt (MCDG-TP) is proposed to complement via controllable classes and ratios. To generalize the text-to-image diffusion model to the proposed MCDG-TP, a class distribution refinement loss is designed as training supervision. For the latter, MCDG-TP in three modes is proposed to synthesize new layout masks from various texts. Our generated data shows significant progress in temporal continuity, spatial diversity, and quality realism, empowering change detectors with accuracy and transferability. The code is available at https://github.com/DZhaoXd/ChangeDiff

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑