Heterogeneous Feature Distillation Network for SAR Image Semantic Segmentation
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences（中国科学院脑科学与智能技术卓越创新中心）
Semantic segmentation for SAR (Synthetic Aperture Radar) images has attracted increasing attention in the remote sensing community recently, due to SAR's all-time and all-weather imaging capability. However, SAR images are generally more difficult to be segmented than their EO (Electro-Optical) counterparts, since speckle noises and layovers are inevitably involved in SAR images. To address this problem, we investigate how to introduce EO features to assist the training of a SAR-segmentation model, and propose a heterogeneous feature distillation network for segmenting SAR images, called HFD-Net, where a SAR-segmentation student model gains knowledge from a pre-trained EO-segmentation teacher model. In the proposed HFD-Net, both the student and teacher models employ an identical architecture but different parameter configurations, and a heterogeneous feature distillation model is explored for transferring latent EO features from the teacher model to the student model and then enhancing the ability of the student model for SAR image segmentation. In addition, a heterogeneous feature alignment module is explored to aggregate multi-scale features for segmentation in each of the student model and teacher model. Extensive experimental results on two public datasets demonstrate that the proposed HFD-Net outperforms seven state-of-the-art SAR image semantic segmentation methods.
Change detection in heterogeneous remote sensing images is crucial for disaster damage assessment. Recent methods use homogenous transformation, which transforms the heterogeneous optical and SAR remote sensing images into the same feature space, to achieve change detection. Such transformations mainly operate on the low-level feature space and may corrupt the semantic content, deteriorating the performance of change detection. To solve this problem, this paper presents a new homogeneous transformation model termed deep homogeneous feature fusion (DHFF) based on image style transfer (IST). Unl…
SARTM: Segment Any RGB Thermal Model with Language aided Distillation
Changchun Institute of Optics, Fine Mechanicsand Physics, Chinese Academy of Sciences University（长春光学精密机械与物理研究所，中国科学院大学）；School of Computing, Macquarie University（麦考瑞大学计算机学院）；School of Computer Science and Informatics, Cardiff University（卡迪夫大学计算机科学与信息学院）；Saw Swee Hock School of Public Health, National University of Singapore（新加坡国立大学 Saw Swee Hock 公共卫生学院）
Beihang University（北京航空航天大学）；SenseTime Research（商汤科技研究院）；ETH Zurich（苏黎世联邦理工学院）；Shandong University（山东大学）；University of Science and Technology of China（中国科学技术大学）
Conventional knowledge distillation (KD) methods for object detection mainly concentrate on homogeneous teacher-student detectors. However, the design of a lightweight detector for deployment is often significantly different from a high-capacity detector. Thus, we investigate KD among heterogeneous teacher-student pairs for a wide application. We observe that the core difficulty for heterogeneous KD (hetero-KD) is the significant semantic gap between the backbone features of heterogeneous detectors due to the different optimization manners. Conventional homogeneous KD (homo-KD) methods suffer …
Socialized Detector Learning: Trajectory-Guided and Reciprocal Distillation for Heterogeneous Object Detectors
School of New Media and Communication, Tianjin University（天津大学新媒体与传播学院）；School of Artificial Intelligence, Tianjin University（天津大学人工智能学院）；School of Computer Science and Technology, Tianjin University（天津大学计算机科学与技术学院）；School of Sports Training, Tianjin University of Sport（天津体育学院运动训练学院）；Faculty of Information Engineering and Automation, Kunming University of Science and Technology（昆明理工大学信息工程与自动化学院）；School of Electronic Science, National University of Defense Technology（国防科技大学电子科学学院）；School of Automation, Southeast University（东南大学自动化学院）；School of Computer Science and Engineering, University of New South Wales（新南威尔士大学计算机科学与工程学院）
Change detection (CD) in remote sensing images has been an ever-expanding area of research. To date, although many methods have been proposed using various techniques, accurately identifying changes is still a great challenge, especially in the high resolution or heterogeneous situations, due to the difficulties in effectively modeling the features from ground objects with different patterns. In this paper, a novel CD method based on the graph convolutional network (GCN) and multiscale object-based technique is proposed for both homogeneous and heterogeneous images. First, the object-wise high…
Shanghai AI Laboratory（上海人工智能实验室）；University of Chinese Academy of Sciences（中国科学院大学）
Knowledge distillation(KD) is a widely-used technique to train compact models in object detection. However, there is still a lack of study on how to distill between heterogeneous detectors. In this paper, we empirically find that better FPN features from a heterogeneous teacher detector can help the student although their detection heads and label assignments are different. However, directly aligning the feature maps to distill detectors suffers from two problems. First, the difference in feature magnitude between the teacher and the student could enforce overly strict constraints on the stude…
Hong Kong University of Science and Technology（香港科技大学）
In this paper, we investigate the knowledge distillation (KD) strategy for object detection and propose an effective framework applicable to both homogeneous and heterogeneous student-teacher pairs. The conventional feature imitation paradigm introduces imitation masks to focus on informative foreground areas while excluding the background noises. However, we find that those methods fail to fully utilize the semantic information in all feature pyramid levels, which leads to inefficiency for knowledge distillation between FPN-based detectors. To this end, we propose a novel semantic-guided feat…
Exploring Generalisability of Self-Distillation with No Labels for SAR-Based Vegetation Prediction
Universitat de València（瓦伦西亚大学）；European Space Agency（欧洲空间局）；Universidad Nacional de Colombia（哥伦比亚国立大学）；University of Cambridge（剑桥大学）；University of Oxford（牛津大学）；Universidad de Antioquia（安蒂奥基亚大学）
Deep learning (DL) in remote sensing has nowadays become an effective operative tool: it is largely used in applications such as change detection, image restoration, segmentation, detection and classification. With reference to synthetic aperture radar (SAR) domain the application of DL techniques is not straightforward due to non trivial interpretation of SAR images, specially caused by the presence of speckle. Several deep learning solutions for SAR despeckling have been proposed in the last few years. Most of these solutions focus on the definition of different network architectures with si…
The increasing intensity and frequency of floods is one of the many consequences of our changing climate. In this work, we explore ML techniques that improve the flood detection module of an operational early flood warning system. Our method exploits an unlabelled dataset of paired multi-spectral and Synthetic Aperture Radar (SAR) imagery to reduce the labeling requirements of a purely supervised learning method. Prior works have used unlabelled data by creating weak labels out of them. However, from our experiments we noticed that such a model still ends up learning the label mistakes in thos…
Heterogeneous SAR-optical fusion for near-real-time land use and land cover mapping under cloud contamination: A novel framework and global benchmark dataset
State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing（信息工程测绘与遥感国家重点实验室）；School of Computer Science and Information Engineering（计算机科学与信息工程学院）；Hangzhou International Innovation Institute（杭州国际创新研究院）；Oriental Space Port Research Institute（东方航天港研究院）；Hubei Luojia Laboratory（湖北珞珈实验室）
Key Laboratory for Information Science of Electromagnetic Waves, Ministry of Education, Fudan University（复旦大学电磁波信息科学教育部重点实验室）；Discipline and Technology Center of Microwave Vision Intelligent Sensing, Fudan University（复旦大学微波视觉智能感知学科与技术中心）
HAD: Heterogeneity-Aware Distillation for Lifelong Heterogeneous Learning
Southern University of Science and Technology（南方科技大学）；Zhejiang University（浙江大学）；Technical University of Munich（慕尼黑工业大学）；City University of Hong Kong（香港城市大学）；Shanghai Jiao Tong University（上海交通大学）
Self-Supervised learning (SSL) has become the new state-of-art in several domain classification and segmentation tasks. Of these, one popular category in SSL is distillation networks such as BYOL. This work proposes RSDnet, which applies the distillation network (BYOL) in the remote sensing (RS) domain where data is non-trivially different from natural RGB images. Since Multi-spectral (MS) and synthetic aperture radar (SAR) sensors provide varied spectral and spatial resolution information, we utilised them as an implicit augmentation to learn invariant feature embeddings. In order to learn RS…
One-for-All: Bridge the Gap Between Heterogeneous Architectures in Knowledge Distillation
School of information and Electronics, Beijing Institute of Technology（北京理工大学信息与电子学院）；Huawei Noah’s Ark Lab（华为诺亚方舟实验室）；School of Computer Science, Faculty of Engineering, The University of Sydney（悉尼大学工程学院计算机科学学院）
Insight SFI Research Centre for Data Analytics（Insight SFI数据分析研究中心）；Dublin City University（都柏林城市大学）；University of Catania（卡塔尼亚大学）
Video saliency prediction has recently attracted attention of the research community, as it is an upstream task for several practical applications. However, current solutions are particularly computationally demanding, especially due to the wide usage of spatio-temporal 3D convolutions. We observe that, while different model architectures achieve similar performance on benchmarks, visual variations between predicted saliency maps are still significant. Inspired by this intuition, we propose a lightweight model that employs multiple simple heterogeneous decoders and adopts several practical app…
Grid-Reg: Detector-Free Gridized Feature Learning and Matching for Large-Scale SAR-Optical Image Registration
Shanghai Key Laboratory of Intelligent Sensing and Recognition（上海智能感知与识别重点实验室）；Shanghai Jiao Tong University（上海交通大学）；Center of Digital Innovation（数字创新中心）
Heterogeneous Knowledge Distillation via Geometry Decoupling and Momentum-Aware Gradient Regulation
School of Computer and Mathematics, Central South University of Forestry and Technology（中南林业科技大学计算机与数学学院）；College of Computer Science and Technology, and the Laboratory of Digitizing Software for Frontier Equipment, National University of Defense Technology（国防科技大学计算机科学与技术学院及前沿装备数字化软件实验室）
The Chinese University of Hong Kong（香港中文大学）；East China Normal University（华东师范大学）；Nanjing University of Posts and Telecommunications（南京邮电大学）
Single image denoising (SID) has achieved significant breakthroughs with the development of deep learning. However, the proposed methods are often accompanied by plenty of parameters, which greatly limits their application scenarios. Different from previous works that blindly increase the depth of the network, we explore the degradation mechanism of the noisy image and propose a lightweight Multiple Degradation and Reconstruction Network (MDRN) to progressively remove noise. Meanwhile, we propose two novel Heterogeneous Knowledge Distillation Strategies (HMDS) to enable MDRN to learn richer an…
Multi-Level Heterogeneous Knowledge Transfer Network on Forward Scattering Center Model for Limited Samples SAR ATR
College of Electronic Science and Technology, National University of Defense Technology（电子科学与技术学院，国防科技大学）；School of Electronic Information, Wuhan University（电子信息学院，武汉大学）
针对有限样本 SAR ATR，本文提出 MHKT 网络，从特征、分布和类别层面将物理可解释的 FSCM 知识迁移到实测 SAR 识别中，并在两个新数据集上验证了优越性能。
Aristotle University of Thessaloniki（塞萨洛尼基亚里士多德大学）
Knowledge Distillation (KD) methods are capable of transferring the knowledge encoded in a large and complex teacher into a smaller and faster student. Early methods were usually limited to transferring the knowledge only between the last layers of the networks, while latter approaches were capable of performing multi-layer KD, further increasing the accuracy of the student. However, despite their improved performance, these methods still suffer from several limitations that restrict both their efficiency and flexibility. First, existing KD methods typically ignore that neural networks undergo…
In this paper, we study the problem of unifying knowledge from a set of classifiers with different architectures and target classes into a single classifier, given only a generic set of unlabelled data. We call this problem Unifying Heterogeneous Classifiers (UHC). This problem is motivated by scenarios where data is collected from multiple sources, but the sources cannot share their data, e.g., due to privacy concerns, and only privately trained models can be shared. In addition, each source may not be able to gather data to train all classes due to data availability at each source, and may n…
Beijing University of Posts and Telecommunications（北京邮电大学）；ByteDance（字节跳动）；USTC（中国科学技术大学）；Beijing Key Laboratory of Network System and Network Culture（北京网络系统与网络文化重点实验室）；Key Laboratory of Interactive Technology and Experience System, Ministry of Culture and Tourism（文化和旅游部互动技术与体验系统重点实验室）；Zhongguancun Academy（中关村科学城）
FedWSIDD: Federated Whole Slide Image Classification via Dataset Distillation
Union Hospital, Tongji Medical College, Huazhong University of Science and Technology（华中科技大学同济医学院附属同济医院）；Centre for Health Informatics, Australian Institute of Health Innovation, Macquarie University（澳大利亚健康创新研究所健康信息中心，麦觉大学）
Technical University of Munich (TUM)（慕尼黑工业大学）；German Aerospace Center (DLR)（德国航空航天中心）
While deep learning techniques have an increasing impact on many technical fields, gathering sufficient amounts of training data is a challenging problem in remote sensing. In particular, this holds for applications involving data from multiple sensors with heterogeneous characteristics. One example for that is the fusion of synthetic aperture radar (SAR) data and optical imagery. With this paper, we publish the SEN1-2 dataset to foster deep learning research in SAR-optical data fusion. SEN1-2 comprises 282,384 pairs of corresponding image patches, collected from across the globe and throughou…
Institute of Computer Science and Technology, Peking University（北京大学计算机科学技术研究所）
Text-to-image synthesis aims to automatically generate images according to text descriptions given by users, which is a highly challenging task. The main issues of text-to-image synthesis lie in two gaps: the heterogeneous and homogeneous gaps. The heterogeneous gap is between the high-level concepts of text descriptions and the pixel-level contents of images, while the homogeneous gap exists between synthetic image distributions and real image distributions. For addressing these problems, we exploit the excellent capability of generic discriminative models (e.g. VGG19), which can guide the tr…
Dealing With Heterogeneous 3D MR Knee Images: A Federated Few-Shot Learning Method With Dual Knowledge Distillation
Rutgers University（罗格斯大学）；Sichuan University West China Hospital（四川大学华西医院）；Walmart Global Tech（沃尔玛全球科技）；Shanghai Jiao Tong University School of Medicine（上海交通大学医学院）；Tongren Hospital（同仁医院）；Fudan University（复旦大学）；Zhongshan Hospital（中山医院）；SenseTime Research（商汤科技研究院）
