arXiv 2410.061942024-10-08cs.CV

基于提示的DirectSAM用于遥感图像语义轮廓提取

Prompting DirectSAM for Semantic Contour Extraction in Remote Sensing Images

College of Computer Science and Software Engineering, Hohai University（河海大学计算机与软件学院）；Hong Kong University of Science and Technology（香港科技大学）；School of Computer and Electronic Information, Nanjing Normal University（南京师范大学计算机与电子信息学院）；School of Information and Communication Technology, Griffith University（格里菲斯大学信息与通信技术学院）

Shiyu Miao, Delong Chen, Fan Liu, Chuanyi Zhang, Yanhui Gu, Shengjie Guo, Jun Zhou

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本研究针对遥感图像语义轮廓提取现有方法依赖小数据集训练专用模型的问题，基于DirectSAM构建集成提示器模块的DirectSAM-RS，搭配超3.4万三元组的大规模数据集，在零样本和微调设置下均达SOTA性能。

摘要：

Direct Segment Anything Model（DirectSAM，直接分割一切模型）在类别无关的轮廓提取任务中表现出色。本文探索了将其应用于光学遥感图像的方法，遥感图像中的语义轮廓提取（如识别建筑物、道路网和海岸线）具有重要的实用价值。目前这类应用依赖于在各领域的小数据集上分别训练专用的小型模型来实现。我们提出了一种基于DirectSAM的基础模型，命名为DirectSAM-RS，它不仅继承了从自然图像中习得的强大分割能力，还受益于我们构建的一个大规模遥感语义轮廓提取数据集。该数据集包含超过3.4万个图像-文本-轮廓三元组，规模至少是单个数据集的30倍。DirectSAM-RS集成了一个提示器模块：在DirectSAM架构上附加了文本编码器和交叉注意力层，可灵活地基于目标类别标签或指代表达式进行条件控制。我们在零样本和微调两种设置下对DirectSAM-RS进行了评估，结果表明它在多个下游基准测试中取得了当前最优的性能。

英文摘要：

The Direct Segment Anything Model (DirectSAM) excels in class-agnostic contour extraction. In this paper, we explore its use by applying it to optical remote sensing imagery, where semantic contour extraction-such as identifying buildings, road networks, and coastlines-holds significant practical value. Those applications are currently handled via training specialized small models separately on small datasets in each domain. We introduce a foundation model derived from DirectSAM, termed DirectSAM-RS, which not only inherits the strong segmentation capability acquired from natural images, but also benefits from a large-scale dataset we created for remote sensing semantic contour extraction. This dataset comprises over 34k image-text-contour triplets, making it at least 30 times larger than individual dataset. DirectSAM-RS integrates a prompter module: a text encoder and cross-attention layers attached to the DirectSAM architecture, which allows flexible conditioning on target class labels or referring expressions. We evaluate the DirectSAM-RS in both zero-shot and fine-tuning setting, and demonstrate that it achieves state-of-the-art performance across several downstream benchmarks.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。