arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2403.13304cs.CV
收藏

DetDiffusion:协同生成与感知模型以增强数据生成与感知

DetDiffusion: Synergizing Generative and Perceptive Models for Enhanced Data Generation and Perception

Tsinghua University(清华大学);CUHK(香港中文大学);HKUST(香港科技大学);Huawei Noah’s Ark Lab(华为诺亚方舟实验室);Research Institute of Tsinghua, Pearl River Delta(清华珠三角研究院)

Yibo Wang, Ruiyuan Gao, Kai Chen, Kaiqiang Zhou, Yingjie Cai, Lanqing Hong, Zhenguo Li, Lihui Jiang, Dit-Yan Yeung, Qiang Xu, Kai Zhang

AI总结:

本文提出 DetDiffusion 框架,首次协同生成与感知模型,通过引入感知感知损失和感知感知属性提升图像生成质量与可控性,并定制数据增强,在目标检测任务中达到 SOTA,显著增强下游检测性能。

摘要:

当前的感知模型严重依赖资源密集型数据集,因此需要创新性解决方案。利用扩散模型的最新进展,通过从各种标注构建图像输入,合成数据被证明对下游任务有益。尽管先前的方法分别处理了生成模型和感知模型,但 DetDiffusion 首次将两者协同,解决了为感知模型生成有效数据的挑战。为了利用感知模型增强图像生成,我们通过分割引入了感知感知损失(P.A. loss),提升了质量与可控性。为了提升特定感知模型的性能,我们的方法通过在生成过程中提取并利用感知感知属性(P.A. Attr)来定制数据增强。目标检测任务的实验结果突显了 DetDiffusion 的卓越性能,在布局引导生成中确立了新的 SOTA。此外,DetDiffusion 的图像合成能有效增强训练数据,显著提升下游检测性能。

英文摘要:

Current perceptive models heavily depend on resource-intensive datasets, prompting the need for innovative solutions. Leveraging recent advances in diffusion models, synthetic data, by constructing image inputs from various annotations, proves beneficial for downstream tasks. While prior methods have separately addressed generative and perceptive models, DetDiffusion, for the first time, harmonizes both, tackling the challenges in generating effective data for perceptive models. To enhance image generation with perceptive models, we introduce perception-aware loss (P.A. loss) through segmentation, improving both quality and controllability. To boost the performance of specific perceptive models, our method customizes data augmentation by extracting and utilizing perception-aware attribute (P.A. Attr) during generation. Experimental results from the object detection task highlight DetDiffusion's superior performance, establishing a new state-of-the-art in layout-guided generation. Furthermore, image syntheses from DetDiffusion can effectively augment training data, significantly enhancing downstream detection performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑