Boosting ship detection in SAR images with complementary pretraining techniques
Qian Xuesen Laboratory of Space Technology, China Academy of Space Technology（中国空间技术研究院钱学森空间技术实验室）；School of Information and Electronics Engineering, Beijing Institute of Technology（北京理工大学信息与电子工程学院）；School of Mathematics and Computational Science, Xiangtan University（湘潭大学数学与计算科学学院）
Wei Bao, Meiyu Huang, Yaqin Zhang, Yao Xu, Xuejiao Liu, Xueshuang Xiang
Deep learning methods have made significant progress in ship detection in synthetic aperture radar (SAR) images. The pretraining technique is usually adopted to support deep neural networks-based SAR ship detectors due to the scarce labeled SAR images. However, directly leveraging ImageNet pretraining is hardly to obtain a good ship detector because of different imaging perspective and geometry. In this paper, to resolve the problem of inconsistent imaging perspective between ImageNet and earth observations, we propose an optical ship detector (OSD) pretraining technique, which transfers the characteristics of ships in earth observations to SAR images from a large-scale aerial image dataset. On the other hand, to handle the problem of different imaging geometry between optical and SAR images, we propose an optical-SAR matching (OSM) pretraining technique, which transfers plentiful texture features from optical images to SAR images by common representation learning on the optical-SAR matching task. Finally, observing that the OSD pretraining based SAR ship detector has a better recall on sea area while the OSM pretraining based SAR ship detector can reduce false alarms on land area, we combine the predictions of the two detectors through weighted boxes fusion to further improve detection results. Extensive experiments on four SAR ship detection datasets and two representative CNN-based detection benchmarks are conducted to show the effectiveness and complementarity of the two proposed detectors, and the state-of-the-art performance of the combination of the two detectors. The proposed method won the sixth place of ship detection in SAR images in 2020 Gaofen challenge.
With the booming of Convolutional Neural Networks (CNNs), CNNs such as VGG-16 and ResNet-50 widely serve as backbone in SAR ship detection. However, CNN based backbone is hard to model long-range dependencies, and causes the lack of enough high-quality semantic information in feature maps of shallow layers, which leads to poor detection performance in complicated background and small-sized ships cases. To address these problems, we propose a SAR ship detection method based on Swin Transformer and Feature Enhancement Feature Pyramid Network (FEFPN). Swin Transformer serves as backbone to model …
A Survey on SAR ship classification using Deep Learning
PhD School in Computer Science, University of Pisa（计算机科学博士学院，比萨大学）；Institute of Information Science and Technologies, National Research Council of Italy（意大利国家研究委员会信息科学与技术研究所）；National Biodiversity Future Center - NBFC（国家生物多样性未来中心 - NBFC）
School of Electronic and Information Engineering, Beihang University（北京航空航天大学电子与信息工程学院）；Edinburgh Napier University（爱丁堡龙比亚大学）；Taibah University（塔伊巴大学）；University of Strathclyde（斯特拉斯克莱德大学）；University of Leicester（莱斯特大学）
Common horizontal bounding box (HBB)-based methods are not capable of accurately locating slender ship targets with arbitrary orientations in synthetic aperture radar (SAR) images. Therefore, in recent years, methods based on oriented bounding box (OBB) have gradually received attention from researchers. However, most of the recently proposed deep learning-based methods for OBB detection encounter the boundary discontinuity problem in angle or key point regression. In order to alleviate this problem, researchers propose to introduce some manually set parameters or extra network branches for di…
OSSDD - a New Open Dataset for Sentinel-1 Ship Detection
Fraunhofer Institute of Optronics, System Technologies and Image Exploitation - IOSB（弗劳恩霍夫光学技术、系统技术与图像利用研究所）；Karlsruhe Institute of Technology - KIT（卡尔斯鲁厄理工学院）；HENSOLDT Sensors GmbH（亨索尔特传感器有限公司）
Nanjing University of Aeronautics and Astronautics（南京航空航天大学）；Collaborative Innovation Center of Novel Software Technology and Industrialization（新型软件技术与产业化协同创新中心）
This paper studies a practically meaningful ship detection problem from synthetic aperture radar (SAR) images by the neural network. We broadly extract different types of SAR image features and raise the intriguing question that whether these extracted features are beneficial to (1) suppress data variations (e.g., complex land-sea backgrounds, scattered noise) of real-world SAR images, and (2) enhance the features of ships that are small objects and have different aspect (length-width) ratios, therefore resulting in the improvement of ship detection. To answer this question, we propose a SAR-s…
University of Electronic Science and Technology of China（电子科技大学）
How to fully utilize polarization to enhance synthetic aperture radar (SAR) ship classification remains an unresolved issue. Thus, we propose a dual-polarization information guided network (DPIG-Net) to solve it.
Qian Xuesen Laboratory of Space Technology, China Academy of Space Technology（中国空间技术研究院钱学森空间技术实验室）
Deep learning techniques have made an increasing impact on the field of remote sensing. However, deep neural networks based fusion of multimodal data from different remote sensors with heterogenous characteristics has not been fully explored, due to the lack of availability of big amounts of perfectly aligned multi-sensor image data with diverse scenes of high resolutions, especially for synthetic aperture radar (SAR) data and optical imagery. To promote the development of deep learning based SAR-optical fusion approaches, we release the QXS-SAROPT dataset, which contains 20,000 pairs of SAR-o…
Huge imbalance of different scenes' sample numbers seriously reduces Synthetic Aperture Radar (SAR) ship detection accuracy. Thus, to solve this problem, this letter proposes a Balance Scene Learning Mechanism (BSLM) for offshore and inshore ship detection in SAR images.
University of Electronic Science and Technology of China（电子科技大学）
Most of existing synthetic aperture radar (SAR) ship in-stance segmentation models do not achieve mask interac-tion or offer limited interaction performance. Besides, their multi-scale ship instance segmentation performance is moderate especially for small ships. To solve these problems, we propose a mask attention interaction and scale enhancement network (MAI-SE-Net) for SAR ship instance segmentation. MAI uses an atrous spatial pyra-mid pooling (ASPP) to gain multi-resolution feature re-sponses, a non-local block (NLB) to model long-range spa-tial dependencies, and a concatenation shuffle a…
Global Fishing Watch（全球渔业观察）；Cambrio；Defense Innovation Unit（国防创新单元）；UC Berkeley（加州大学伯克利分校）
Unsustainable fishing practices worldwide pose a major threat to marine resources and ecosystems. Identifying vessels that do not show up in conventional monitoring systems -- known as ``dark vessels'' -- is key to managing and securing the health of marine environments. With the rise of satellite-based synthetic aperture radar (SAR) imaging and modern machine learning (ML), it is now possible to automate detection of dark vessels day or night, under all-weather conditions. SAR images, however, require a domain-specific treatment and are not widely accessible to the ML community. Maritime obje…
Shape deformation of targets in SAR image due to random orientation and partial information loss caused by occlusion of the radar signal, is an essential challenge in SAR ship detection. In this paper, we propose a data augmentation method to train a deep network that is robust to partial information loss within the targets. Taking advantage of ground-truth annotations for bounding box and instance segmentation mask, we present a simple and effective pipeline to simulate information loss on targets in instance-level, while preserving contextual information. Furthermore, we adopt deformable con…
Cross-modal Ship Re-Identification via Optical and SAR Imagery: A Novel Dataset and Method
Technology and Engineering Center for Space Utilization, Chinese Academy of Sciences（中国科学院空间利用技术与工程中心）；University of Chinese Academy of Sciences（中国科学院大学）
MOS: Mitigating Optical-SAR Modality Gap for Cross-Modal Ship Re-Identification
School of Artificial Intelligence, Beihang University（北京航空航天大学人工智能学院）；School of Computer Science and Engineering, Beihang University（北京航空航天大学计算机科学与工程学院）
SDF-Net: Structure-Aware Disentangled Feature Learning for Opticall-SAR Ship Re-identification
Technology and Engineering Center for Space Utilization, Chinese Academy of Sciences（中国科学院空间利用技术与工程中心）；Key Laboratory of Space Utilization, Chinese Academy of Sciences（中国科学院空间利用重点实验室）；University of Chinese Academy of Sciences（中国科学院大学）；School of Software, Beihang University（北航软件学院）
Synthetic Aperture Radar (SAR) constitutes a fundamental asset for wide-areas monitoring with high-resolution requirements. The first SAR sensors have given rise to coarse coastal and maritime monitoring applications, including oil spill, ship and ice floes detection. With the upgrade to very high-resolution sensors in the recent years, with relatively new SAR missions such as Sentinel-1, a great deal of data providing a stronger information content has been released, enabling more refined studies on general targets features and thus permitting complex classifications, as for ship classificati…
The detection and prevention of illegal fishing is critical to maintaining a healthy and functional ecosystem. Recent research on ship detection in satellite imagery has focused exclusively on performance improvements, disregarding detection efficiency. However, the speed and compute cost of vessel detection are essential for a timely intervention to prevent illegal fishing. Therefore, we investigated optimization methods that lower detection time and cost with minimal performance loss. We trained an object detection model based on a convolutional neural network (CNN) using a dataset of satell…
Seg-CycleGAN : SAR-to-optical image translation guided by a downstream task
Gina Cody School of Engineering and Computer Science, Concordia University（康考迪亚大学吉娜·科迪工程与计算机科学学院）；School of Automation, Northwestern Polytechnical University（西北工业大学自动化学院）；School of Cybersecurity, Northwestern Polytechnical University（西北工业大学网络空间安全学院）
Domain Adaptive SAR Wake Detection: Leveraging Similarity Filtering and Memory Guidance
College of Computer Science and Technology, Qingdao University（青岛大学计算机科学与技术学院）；Laboratory for Regional Oceanography and Numerical Modeling, Qingdao Marine Science and Technology Center（青岛海洋科学与技术中心区域海洋学与数值模拟实验室）；School of Computer Science and Information Technology, The University of Nottingham（诺丁汉大学计算机科学与信息学院）；State Key Laboratory of Physical Oceanography, Department of Marine Technology, Ocean University of China（中国海洋大学物理海洋学国家重点实验室，海洋技术系）
In this paper, we present a ship detection pipeline for low-cost medium resolution satellite optical imagery obtained from ESA Sentinel-2 and Planet Labs Dove constellations. This optical satellite imagery is readily available for any place on Earth and underutilized in the maritime domain, compared to existing solutions based on synthetic-aperture radar (SAR) imagery. We developed a ship detection method based on a state-of-the-art deep-learning-based object detection method which was developed and evaluated on a large-scale dataset that was collected and automatically annotated with the help…
Deep learning ship detection in satellite optical imagery suffers from false positive occurrences with clouds, landmasses, and man-made objects that interfere with correct classification of ships, typically limiting class accuracy scores to 88\%. This work explores the tensions between customization strategies, class accuracy rates, training times, and costs in cloud based solutions. We demonstrate how a custom U-Net can achieve 92\% class accuracy over a validation dataset and 68\% over a target dataset with 90\% confidence. We also compare a single node architecture with a parameter server v…
Avinashilingam University for Women（阿维纳什林加姆女子大学）；Naval Research Board-DRDO（海军研究委员会-国防研究与发展组织）
A new fangled method for ship wake detection in synthetic aperture radar (SAR) images is explored here. Most of the detection procedure applies the Radon transform as its properties outfit more than any other transformation for the detection purpose. But still it holds problems when the transform is applied to an image with a high level of noise. Here this paper articulates the combination between the radon transformation and the shrinkage methods which increase the mode of wake detection process. The latter shrinkage method with RT maximize the signal to noise ratio hence it leads to most opt…
FUELVISION: A Multimodal Data Fusion and Multimodel Ensemble Algorithm for Wildfire Fuels Mapping
University of California Los Angeles（加利福尼亚大学洛杉矶分校）；University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Desert Research Institute（沙漠研究所）；Amazon（亚马逊公司）；United States Forest Service（美国林务局）
本文提出 FUELVISION 算法，融合多源遥感与地形数据，结合生成式 AI 合成数据增强训练集，采用多模型集成方法实现近实时野火可燃物制图，准确率近 80%。
Bridging Modalities and Tasks: A Unified Hierarchical ViT for SAR-to-Optical Translation and Semantic Segmentation
School of Automation, Northwestern Polytechnical University（西北工业大学自动化学院）；School of Cybersecurity, Northwestern Polytechnical University（西北工业大学网络空间安全学院）
