arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2307.12018cs.CV
收藏

GEM:借助视觉基础模型提升用于玻璃表面分割的简单网络性能

GEM: Boost Simple Network for Glass Surface Segmentation via Vision Foundation Models

School of Mechanical Science and Engineering, Huazhong University of Science and Technology(华中科技大学机械科学与工程学院);University of Hongkong(香港大学)

Jing Hao, Moyun Liu, Jinrong Yang, Kuo Feng Hung

AI总结:

针对玻璃表面检测现有方案数据集构建成本高、模型复杂的问题,研究借助Stable Diffusion构建零成本合成数据集S-GSD,基于SAM设计简单分割器GEM,在GSD-S数据集上取得新SOTA,且合成数据集在零样本和迁移学习中表现优异。

摘要:

由于玻璃区域的透明性和反射特性存在固有模糊性,检测玻璃区域是一项具有挑战性的任务。该领域当前的解决方案仍依赖传统深度学习范式,需要构建标注数据集并设计网络架构。但这些主流方案的明显缺陷在于数据集整理过程耗时耗力,且模型结构日益复杂。本文提出充分利用两种现有视觉基础模型(VFM)——Stable Diffusion(稳定扩散模型)和Segment Anything Model(SAM,分割一切模型)的能力来解决上述问题。首先,我们通过Stable Diffusion在无需任何人力成本的情况下,构建了一个合成但具有照片级真实感的大规模玻璃表面检测数据集,命名为S-GSD。该数据集包含4种不同尺度,总计16.8万张图像并配有精确掩码。此外,基于SAM强大的分割能力,我们设计了一个名为GEM的简单玻璃表面分割器,它采用基于查询的简单编码器-解码器架构。我们在大规模玻璃分割数据集GSD-S上开展了全面实验。在这两种视觉基础模型的助力下,我们的GEM取得了新的最优性能,相比已报道的最佳方法GlassSemNet,交并比(IoU)提升了2.1%。此外,大量实验表明,我们的合成数据集S-GSD在零样本和迁移学习场景下表现出优异性能。代码、数据集和模型已公开发布于:https://github.com/isbrycee/GEM

英文摘要:

Detecting glass regions is a challenging task due to the inherent ambiguity in their transparency and reflective characteristics. Current solutions in this field remain rooted in conventional deep learning paradigms, requiring the construction of annotated datasets and the design of network architectures. However, the evident drawback with these mainstream solutions lies in the time-consuming and labor-intensive process of curating datasets, alongside the increasing complexity of model structures. In this paper, we propose to address these issues by fully harnessing the capabilities of two existing vision foundation models (VFMs): Stable Diffusion and Segment Anything Model (SAM). Firstly, we construct a Synthetic but photorealistic large-scale Glass Surface Detection dataset, dubbed S-GSD, without any labour cost via Stable Diffusion. This dataset consists of four different scales, consisting of 168k images totally with precise masks. Besides, based on the powerful segmentation ability of SAM, we devise a simple Glass surface sEgMentor named GEM, which follows the simple query-based encoder-decoder architecture. Comprehensive experiments are conducted on the large-scale glass segmentation dataset GSD-S. Our GEM establishes a new state-of-the-art performance with the help of these two VFMs, surpassing the best-reported method GlassSemNet with an IoU improvement of 2.1%. Additionally, extensive experiments demonstrate that our synthetic dataset S-GSD exhibits remarkable performance in zero-shot and transfer learning settings. Codes, datasets and models are publicly available at: https://github.com/isbrycee/GEM

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑