针对海洋动物分割中长程特征与像素连通性不足、SAM 缺乏水下先验的问题，提出 Dual-SAM 框架，结合多层耦合提示、扩张融合注意力和十字交叉连通性预测，在五个 MAS 数据集上取得最优性能。
摘要：
作为水下智能的重要支柱，海洋动物分割（Marine Animal Segmentation, MAS）涉及在海洋环境中分割动物。以往的方法不擅长提取长程上下文特征，并且忽视了离散像素之间的连通性。近年来，Segment Anything Model（SAM）为通用分割任务提供了一个通用框架。遗憾的是，由于 SAM 是用自然图像训练的，它没有获得来自海洋图像的先验知识。此外，SAM 的单位置提示对于先验引导非常不足。为了解决这些问题，我们提出了一种新颖的特征学习框架，称为 Dual-SAM，用于高性能 MAS。为此，我们首先引入了一种具有 SAM 范式的双结构，以增强海洋图像的特征学习。然后，我们提出了一种多层耦合提示（Multi-level Coupled Prompt, MCP）策略，以注入全面的水下先验信息，并通过适配器增强 SAM 编码器的多层特征。随后，我们设计了一个扩张融合注意力模块（Dilated Fusion Attention Module, DFAM），以逐步整合来自 SAM 编码器的多层特征。最后，我们不是直接预测海洋动物的掩码，而是提出了一种十字交叉连通性预测（Criss-Cross Connectivity Prediction, C$^3$P）范式，以捕获离散像素之间的相互连通性。借助双解码器，它生成伪标签并实现互补特征表示的相互监督，从而相比以往技术取得了显著改进。大量实验验证了我们提出的方法在五个广泛使用的 MAS 数据集上达到了最先进的性能。代码可在 https://github.com/Drchip61/Dual_SAM 获取。
英文摘要：
As an important pillar of underwater intelligence, Marine Animal Segmentation (MAS) involves segmenting animals within marine environments. Previous methods don't excel in extracting long-range contextual features and overlook the connectivity between discrete pixels. Recently, Segment Anything Model (SAM) offers a universal framework for general segmentation tasks. Unfortunately, trained with natural images, SAM does not obtain the prior knowledge from marine images. In addition, the single-position prompt of SAM is very insufficient for prior guidance. To address these issues, we propose a novel feature learning framework, named Dual-SAM for high-performance MAS. To this end, we first introduce a dual structure with SAM's paradigm to enhance feature learning of marine images. Then, we propose a Multi-level Coupled Prompt (MCP) strategy to instruct comprehensive underwater prior information, and enhance the multi-level features of SAM's encoder with adapters. Subsequently, we design a Dilated Fusion Attention Module (DFAM) to progressively integrate multi-level features from SAM's encoder. Finally, instead of directly predicting the masks of marine animals, we propose a Criss-Cross Connectivity Prediction (C$^3$P) paradigm to capture the inter-connectivity between discrete pixels. With dual decoders, it generates pseudo-labels and achieves mutual supervision for complementary feature representations, resulting in considerable improvements over previous techniques. Extensive experiments verify that our proposed method achieves state-of-the-art performances on five widely-used MAS datasets. The code is available at https://github.com/Drchip61/Dual_SAM.
New York University（纽约大学）；University of Southern California（南加州大学）；Clarifai（Clarifai公司）
The ability to identify the same person from multiple camera views without the explicit use of facial recognition is receiving commercial and academic interest. The current status-quo solutions are based on attention neural models. In this paper, we propose Attention and CL loss, which is a hybrid of center and Online Soft Mining (OSM) loss added to the attention loss on top of a temporal attention-based neural network. The proposed loss function applied with bag-of-tricks for training surpasses the state of the art on the common person Re-ID datasets, MARS and PRID 2011. Our source code is pu…
HFP-SAM: Hierarchical Frequency Prompted SAM for Efficient Marine Animal Segmentation
School of Future Technology（未来技术学院）；Central Hospital of Dalian University of Technology（大连理工大学附属中心医院）；Affiliated Zhongshan Hospital of Dalian University（大连大学附属中山医院）；School of Information and Communication Engineering（信息与通信工程学院）
SAMSelect: A Spectral Index Search for Marine Debris Visualization using Segment Anything
Laboratory of Geo-information Science and Remote Sensing, Wageningen University（地理信息科学与遥感实验室，瓦赫宁根大学）；Fundamental AI Lab at the University of Technology Nuremberg（技术大学纽伦堡的基础人工智能实验室）
本文提出 SAMSelect 算法，利用 Segment Anything Model 在小型标注数据上搜索最佳三通道波段或光谱指数组合，以增强 Sentinel-2 海洋垃圾目视解译，并在多地场景中验证其优于文献指数。
Garbage and waste disposal is one of the biggest challenges currently faced by mankind. Proper waste disposal and recycling is a must in any sustainable community, and in many coastal areas there is significant water pollution in the form of floating or submerged garbage. This is called marine debris. Submerged marine debris threatens marine life, and for shallow coastal areas, it can also threaten fishing vessels [Iñiguez et al. 2016, Renewable and Sustainable Energy Reviews]. Submerged marine debris typically stays in the environment for a long time (20+ years), and consists of materials tha…
School of Computer Science, Wuhan University（武汉大学计算机学院）；School of Computer Science, The University of Sydney（悉尼大学计算机学院）；School of Computer Science and Engineering, Nanyang Technological University（南洋理工大学计算机科学与工程学院）
Institute of Natural and Technical Systems（自然与技术系统研究所）；Sevastopol State University（塞瓦斯托波尔国立大学）
The efficiency of using the YOLOV5 machine learning model for solving the problem of automatic de-tection and recognition of micro-objects in the marine environment is studied. Samples of microplankton and microplastics were prepared, according to which a database of classified images was collected for training an image recognition neural network. The results of experiments using a trained network to find micro-objects in photo and video images in real time are presented. Experimental studies have shown high efficiency, comparable to manual recognition, of the proposed model in solving problem…
Norweigan University of Science and Technology（挪威科技大学）；SINTEF Digital（挪威科技工业研究院数字部）；Oklahoma State University（俄克拉荷马州立大学）；Kongsberg Maritime（康士伯海事公司）
With the rise of focus on man made changes to our planet and wildlife therein, more and more emphasis is put on sustainable and responsible gathering of resources. In an effort to preserve maritime wildlife the Norwegian government has decided that it is necessary to create an overview over the presence and abundance of various species of wildlife in the Norwegian fjords and oceans. In this paper we apply and analyze an object detection scheme that detects fish in camera images. The data is sampled from a submerged data station at Fulehuk in Norway. We implement You Only Look Once (YOLO) versi…
FishAI 2.0: Marine Fish Image Classification with Multi-modal Few-shot Learning
Institute of Image Processing and Pattern Recognition, Shanghai Jiao Tong University（上海交通大学图像处理与模式识别研究所）；Key Laboratory of System Control and Information Processing, Ministry of Education of China（中国教育部系统控制与信息处理重点实验室）；State Key Laboratory of Submarine Geoscience, Second Institute of Oceanography, Ministry of Natural Resources（自然资源部第二海洋研究所海底科学国家重点实验室）；Key Laboratory of Marine Ecosystem Dynamics, Ministry of Natural Resources（自然资源部海洋生态系统动力学重点实验室）；Second Institute of Oceanography, Ministry of Natural Resources（自然资源部第二海洋研究所）
How many labels do you need? A decision framework for cross-habitat marine species recognition
College of Science and Engineering, James Cook University（詹姆斯·库克大学科学与工程学院）；Centre for AI and Data Science Innovation, James Cook University（詹姆斯·库克大学人工智能与数据科学创新中心）
German Research Center for Artificial Intelligence（德国人工智能研究中心）；Bonn-Rhein-Sieg University of Applied Sciences（波恩-莱茵-锡格应用技术大学）
Machine learning and neural networks are now ubiquitous in sonar perception, but it lags behind the computer vision field due to the lack of data and pre-trained models specifically for sonar images. In this paper we present the Marine Debris Turntable dataset and produce pre-trained neural networks trained on this dataset, meant to fill the gap of missing pre-trained models for sonar images. We train Resnet 20, MobileNets, DenseNet121, SqueezeNet, MiniXception, and an Autoencoder, over several input image sizes, from 32 x 32 to 96 x 96, on the Marine Debris turntable dataset. We evaluate thes…
Advancing Marine Research: UWSAM Framework and UIIS10K Dataset for Precise Underwater Instance Segmentation
School of Computer Science and Technology, Hainan University（海南大学计算机科学与技术学院）；School of Computer Science and Technology, Huazhong University of Science and Technology（华中科技大学计算机科学与技术学院）；School of Control Science and Engineering, Shandong University（山东大学控制科学与工程学院）；College of Computer Science, Nankai University（南开大学计算机学院）；School of Computer Science and Artificial Intelligence, Zhengzhou University（郑州大学计算机科学与人工智能学院）；Department of Automation, Shanghai Jiao Tong University（上海交通大学自动化学院）；School of Data Science, Lingnan University（岭南大学数据科学学院）
针对 SAM 在水下实例分割中的领域适配与计算开销问题，本文构建 UIIS10K 数据集，并提出结合 MG-UKD 与 EUPG 的高效 UWSAM，在多个数据集上显著优于现有方法。
SAM2-UNet: Segment Anything 2 Makes Strong Encoder for Natural and Medical Image Segmentation
School of Computer Science and Engineering, Sun Yat-sen University（中山大学计算机科学与工程学院）；School of Computer and Information Engineering, Jiangxi Normal University（江西师范大学计算机与信息工程学院）；The Chinese University of Hong Kong (Shenzhen)（香港中文大学（深圳））；Tianjin University（天津大学）；Monash University（墨尔本大学）；Pazhou Lab（琶洲实验室）；Smart Hospital Research Institute, Peking University Shenzhen Hospital（北京大学深圳医院智能医院研究所）
