arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2401.15282cs.CV
收藏

GEM:通过Segment Anything模型和数据合成提升玻璃表面分割的简单网络

GEM: Boost Simple Network for Glass Surface Segmentation via Segment Anything Model and Data Synthesis

Huazhong University of Science and Technology(华中科技大学);The University of Hong Kong(香港大学)

Jing Hao, Moyun Liu, Kuo Feng Hung

AI总结:

针对玻璃表面分割因透明反射特性导致的模糊性问题,提出仅含SAM骨干等简单结构的GEM分割器,结合SAM和Stable Diffusion模型及合成数据集S-GSD,在GSD-S验证集上IoU提升2.1%。

摘要:

由于玻璃的透明性和反射特性具有模糊性,检测玻璃区域是一项具有挑战性的任务。这些透明玻璃同时具有透射的任意背景场景和反射物体的视觉外观,因此没有固定模式。最近的视觉基础模型在海量数据上训练,在图像感知和生成方面表现出惊人性能。为了更准确地分割玻璃表面,我们充分利用两个视觉基础模型:Segment Anything(SAM,任意分割模型)和Stable Diffusion(稳定扩散模型)。具体而言,我们设计了一个名为GEM的简单玻璃表面分割器,仅包含SAM骨干网络、简单特征金字塔和敏锐查询选择模块以及掩码解码器。敏锐查询选择模块能自适应识别玻璃表面特征,并将其作为掩码解码器的初始查询。我们还通过扩散模型提出了一个合成但逼真的大规模玻璃表面检测数据集S-GSD,包含原始真实数据规模的1倍、5倍、10倍和20倍。该数据集是迁移学习的可行来源。合成数据的规模对迁移学习有积极影响,但随着数据量增加,改进会逐渐饱和。大量实验表明,GEM在GSD-S验证集上达到了新的最先进性能(IoU提升2.1%)。代码和数据集可在https://github.com/isbrycee/GEM-Glass-Segmentor获取。

英文摘要:

Detecting glass regions is a challenging task due to the ambiguity of their transparency and reflection properties. These transparent glasses share the visual appearance of both transmitted arbitrary background scenes and reflected objects, thus having no fixed patterns.Recent visual foundation models, which are trained on vast amounts of data, have manifested stunning performance in terms of image perception and image generation. To segment glass surfaces with higher accuracy, we make full use of two visual foundation models: Segment Anything (SAM) and Stable Diffusion.Specifically, we devise a simple glass surface segmentor named GEM, which only consists of a SAM backbone, a simple feature pyramid, a discerning query selection module, and a mask decoder. The discerning query selection can adaptively identify glass surface features, assigning them as initialized queries in the mask decoder. We also propose a Synthetic but photorealistic large-scale Glass Surface Detection dataset dubbed S-GSD via diffusion model with four different scales, which contain 1x, 5x, 10x, and 20x of the original real data size. This dataset is a feasible source for transfer learning. The scale of synthetic data has positive impacts on transfer learning, while the improvement will gradually saturate as the amount of data increases. Extensive experiments demonstrate that GEM achieves a new state-of-the-art on the GSD-S validation set (IoU +2.1%). Codes and datasets are available at: https://github.com/isbrycee/GEM-Glass-Segmentor.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑