DreamMask：利用合成数据提升开放词汇全景分割
DreamMask: Boosting Open-vocabulary Panoptic Segmentation with Synthetic Data
AI总结：
提出DreamMask框架，通过自动数据生成管道和合成-真实对齐损失，利用合成数据提升开放词汇全景分割模型对新类别的泛化能力，在多个基准上显著超越现有方法。
摘要：
开放词汇全景分割因其在现实世界中的适用性而受到广泛关注。尽管声称具有鲁棒的泛化能力，我们发现先前工作的进步主要归因于已训练类别，暴露出对新颖类别缺乏泛化能力。在本文中，我们从以数据为中心的角度探索提升现有模型。我们提出了DreamMask，系统地探索如何在开放词汇设置中生成训练数据，以及如何使用真实和合成数据训练模型。对于第一部分，我们提出了一种使用现成模型的自动数据生成管道。我们提出了词汇扩展、布局安排、数据过滤等关键设计。配备了这些技术，我们生成的数据可以显著优于手动收集的网络数据。为了使用生成的数据训练模型，设计了一种合成-真实对齐损失来弥合表示差距，在多个基准测试中带来了显著改进。总体而言，DreamMask显著简化了大规模训练数据的收集，作为现有方法的即插即用增强。例如，当在COCO上训练并在ADE20K上测试时，配备DreamMask的模型以2.1% mIoU的显著优势超越了先前的最先进方法。
英文摘要：
Open-vocabulary panoptic segmentation has received significant attention due to its applicability in the real world. Despite claims of robust generalization, we find that the advancements of previous works are attributed mainly on trained categories, exposing a lack of generalization to novel classes. In this paper, we explore boosting existing models from a data-centric perspective. We propose DreamMask, which systematically explores how to generate training data in the open-vocabulary setting, and how to train the model with both real and synthetic data. For the first part, we propose an automatic data generation pipeline with off-the-shelf models. We propose crucial designs for vocabulary expansion, layout arrangement, data filtering, etc. Equipped with these techniques, our generated data could significantly outperform the manually collected web data. To train the model with generated data, a synthetic-real alignment loss is designed to bridge the representation gap, bringing noticeable improvements across multiple benchmarks. In general, DreamMask significantly simplifies the collection of large-scale training data, serving as a plug-and-play enhancement for existing methods. For instance, when trained on COCO and tested on ADE20K, the model equipped with DreamMask outperforms the previous state-of-the-art by a substantial margin of 2.1% mIoU.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
