Nanjing University of Aeronautics and Astronautics（南京航空航天大学）；Shenzhen Research Institute, Nanjing University of Aeronautics and Astronautics（南京航空航天大学深圳研究院）；Hong Kong Polytechnic University（香港理工大学）；Lingnan University（岭南大学）；Hong Kong Metropolitan University（香港都会大学）
Jie Wang, Yongzhen Wang, Yidan Feng, Lina Gong, Xuefeng Yan, Haoran Xie, Fu Lee Wang, Mingqiang Wei
Image smoothing is a fundamental low-level vision task that aims to preserve salient structures of an image while removing insignificant details. Deep learning has been explored in image smoothing to deal with the complex entanglement of semantic structures and trivial details. However, current methods neglect two important facts in smoothing: 1) naive pixel-level regression supervised by the limited number of high-quality smoothing ground-truth could lead to domain shift and cause generalization problems towards real-world images; 2) texture appearance is closely related to object semantics, so that image smoothing requires awareness of semantic difference to apply adaptive smoothing strengths. To address these issues, we propose a novel Contrastive Semantic-Guided Image Smoothing Network (CSGIS-Net) that combines both contrastive prior and semantic prior to facilitate robust image smoothing. The supervision signal is augmented by leveraging undesired smoothing effects as negative teachers, and by incorporating segmentation tasks to encourage semantic distinctiveness. To realize the proposed network, we also enrich the original VOC dataset with texture enhancement and smoothing labels, namely VOC-smooth, which first bridges image smoothing and semantic segmentation. Extensive experiments demonstrate that the proposed CSGIS-Net outperforms state-of-the-art algorithms by a large margin. Code and dataset are available at https://github.com/wangjie6866/CSGIS-Net.
Language-guided Medical Image Segmentation with Target-informed Multi-level Contrastive Alignments
School of Computer Science, The University of Sydney（悉尼大学计算机科学学院）；Institute of Translational Medicine, Shanghai Jiao Tong University（上海交通大学转化医学研究院）；Zhongguancun Academy & Zhongguancun Institute of Artificial Intelligence, Beijing, China（北京中关村学院及中关村人工智能研究院）；Department of Molecular Imaging, Royal Prince Alfred Hospital（皇家珀斯阿尔弗雷德医院分子成像部）
University of Technology Sydney（悉尼科技大学）；Wenzhou University（温州大学）
Zero-shot object detection (ZSD), the task that extends conventional detection models to detecting objects from unseen categories, has emerged as a new challenge in computer vision. Most existing approaches tackle the ZSD task with a strict mapping-transfer strategy, which may lead to suboptimal ZSD results: 1) the learning process of those models ignores the available unseen class information, and thus can be easily biased towards the seen categories; 2) the original visual feature space is not well-structured and lack of discriminative information. To address these issues, we develop a novel…
The Chinese University of Hong Kong（香港中文大学）；SmartMore（思谋科技）
Rapid progress in 3D semantic segmentation is inseparable from the advances of deep network models, which highly rely on large-scale annotated data for training. To address the high cost and challenges of 3D point-level labeling, we present a method for semi-supervised point cloud semantic segmentation to adopt unlabeled point clouds in training to boost the model performance. Inspired by the recent contrastive loss in self-supervised tasks, we propose the guided point contrastive loss to enhance the feature representation and model generalization ability in semi-supervised setting. Semantic p…
Hong Kong University of Science and Technology（香港科技大学）
In this paper, we investigate the knowledge distillation (KD) strategy for object detection and propose an effective framework applicable to both homogeneous and heterogeneous student-teacher pairs. The conventional feature imitation paradigm introduces imitation masks to focus on informative foreground areas while excluding the background noises. However, we find that those methods fail to fully utilize the semantic information in all feature pyramid levels, which leads to inefficiency for knowledge distillation between FPN-based detectors. To this end, we propose a novel semantic-guided feat…
SuperCL: Superpixel Guided Contrastive Learning for Medical Image Segmentation Pre-training
Institute of Medical Technology, Peking University Health Science Center, Peking University（北京大学医学部医学技术研究所）；Department of Biomedical Engineering, Peking University（北京大学生物医学工程系）；National Biomedical Imaging Center, Peking University（北京大学国家生物医学成像中心）；Institute of Biomedical Engineering, Shenzhen Bay Laboratory（深圳湾实验室生物医学工程研究所）；Institute of Biomedical Engineering, Peking University Shenzhen Graduate School（北京大学深圳研究生院生物医学工程研究所）
Beijing University of Posts and Telecommunications（北京邮电大学）
Unsupervised person re-identification (Re-ID) aims to match pedestrian images from different camera views in unsupervised setting. Existing methods for unsupervised person Re-ID are usually built upon the pseudo labels from clustering. However, the quality of clustering depends heavily on the quality of the learned features, which are overwhelmingly dominated by the colors in images especially in the unsupervised setting. In this paper, we propose a Cluster-guided Asymmetric Contrastive Learning (CACL) approach for unsupervised person Re-ID, in which cluster structure is leveraged to guide the…
Recently, adversarial training has been incorporated in self-supervised contrastive pre-training to augment label efficiency with exciting adversarial robustness. However, the robustness came at a cost of expensive adversarial training. In this paper, we show a surprising fact that contrastive pre-training has an interesting yet implicit connection with robustness, and such natural robustness in the pre trained representation enables us to design a powerful robust algorithm against adversarial attacks, RUSH, that combines the standard contrastive pre-training and randomized smoothing. It boost…
University of Science and Technology of China（中国科学技术大学）
Contrastive learning (CL) is a form of self-supervised learning and has been widely used for various tasks. Different from widely studied instance-level contrastive learning, pixel-wise contrastive learning mainly helps with pixel-wise tasks such as medical landmark detection. The counterpart to an instance in instance-level CL is a pixel, along with its neighboring context, in pixel-wise CL. Aiming to build better feature representation, there is a vast literature about designing instance augmentation strategies for instance-level CL; but there is little similar work on pixel augmentation for…
Harbin Institute of Technology at Shenzhen（哈尔滨工业大学（深圳））；The Chinese University of Hong Kong at Shenzhen（香港中文大学（深圳））
Typical methods for pedestrian detection focus on either tackling mutual occlusions between crowded pedestrians, or dealing with the various scales of pedestrians. Detecting pedestrians with substantial appearance diversities such as different pedestrian silhouettes, different viewpoints or different dressing, remains a crucial challenge. Instead of learning each of these diverse pedestrian appearance features individually as most existing methods do, we propose to perform contrastive learning to guide the feature learning in such a way that the semantic distance between pedestrians with diffe…
Prior-guided Hierarchical Instance-pixel Contrastive Learning for Ultrasound Speckle Noise Suppression
School of Biological Science and Medical Engineering, Southeast University（生物科学与医学工程学院，东南大学）；Department of Biomedical Engineering, The Ohio State University（生物医学工程系，奥斯特大学）；Jiangsu Key Laboratory of Biomaterials and Devices, Southeast University（江苏省生物材料与器件重点实验室，东南大学）；State Key Laboratory of Digital Medical Engineering, Southeast University（国家数字医学工程重点实验室，东南大学）
Deep clustering aims to learn a clustering representation through deep architectures. Most of the existing methods usually conduct clustering with the unique goal of maximizing clustering performance, that ignores the personalized demand of clustering tasks.% and results in unguided clustering solutions. However, in real scenarios, oracles may tend to cluster unlabeled data by exploiting distinct criteria, such as distinct semantics (background, color, object, etc.), and then put forward personalized clustering tasks. To achieve task-aware clustering results, in this study, Oracle-guided Contr…
Video-based Generalized Category Discovery via Memory-Guided Consistency-Aware Contrastive Learning
Laboratory for Big Data and Decision（大数据与决策实验室）；National University of Defense Technology（国防科技大学）；Department of Information Engineering and Computer Science（信息工程与计算机科学系）
Generalizable Synthetic Image Detection via Language-guided Contrastive Learning
School of Computer Science and Engineering, University of Electronic Science and Technology of China（电子科技大学计算机科学与工程学院）；State Key Laboratory of Internet of Things for Smart City（智慧城市物联网国家重点实验室）；Department of Computer and Information Science, Faculty of Science and Technology, University of Macau（澳门大学计算机与信息科学系）；Faculty of Applied Sciences, Macao Polytechnic University（澳门 polytechnic 大学应用科学系）
Self-supervised learning holds promise in leveraging large numbers of unlabeled data. However, its success heavily relies on the highly-curated dataset, e.g., ImageNet, which still needs human cleaning. Directly learning representations from less-curated scene images is essential for pushing self-supervised learning to a higher level. Different from curated images which include simple and clear semantic information, scene images are more complex and mosaic because they often include complex scenes and multiple objects. Despite being feasible, recent works largely overlooked discovering the mos…
Although an object may appear in numerous contexts, we often describe it in a limited number of ways. Language allows us to abstract away visual variation to represent and communicate concepts. Building on this intuition, we propose an alternative approach to visual representation learning: using language similarity to sample semantically similar image pairs for contrastive learning. Our approach diverges from image-based contrastive learning by sampling view pairs using language similarity instead of hand-crafted augmentations or learned clusters. Our approach also differs from image-text con…
Sensetime Research（商汤科技研究院）；The Hong Kong Polytechnic University（香港理工大学）
Video highlight detection is a crucial yet challenging problem that aims to identify the interesting moments in untrimmed videos. The key to this task lies in effective video representations that jointly pursue two goals, \textit{i.e.}, cross-modal representation learning and fine-grained feature discrimination. In this paper, these two challenges are tackled by not only enriching intra-modality and cross-modality relations for representation modeling but also shaping the features in a discriminative manner. Our proposed method mainly leverages the intra-modality encoding and cross-modality co…
Samsung Semiconductor US（美国三星半导体）；National Yang Ming Chiao Tung University（国立阳明交通大学）
Point cloud data plays an essential role in robotics and self-driving applications. Yet, annotating point cloud data is time-consuming and nontrivial while they enable learning discriminative 3D representations that empower downstream tasks, such as classification and segmentation. Recently, contrastive learning-based frameworks have shown promising results for learning 3D representations in a self-supervised manner. However, existing contrastive learning methods cannot precisely encode and associate structural features and search the higher dimensional augmentation space efficiently. In this …
Clustering-Guided Multi-Layer Contrastive Representation Learning for Citrus Disease Classification
College of Informatics, Huazhong Agricultural University（华中农业大学信息学院）；Huazhong Agricultural University（华中农业大学）；Engineering Research Center of Intelligent Technology for Agriculture, Ministry of Education（农业智能技术工程研究中心）；National Key Laboratory for Germplasm Innovation & Utilization of Horticultural Crops（园艺作物种质创新与利用国家重点实验室）
Shenzhen University（深圳大学）；University of Leeds（利兹大学）；KU Leuven（鲁汶大学）；Luohu People’s Hospital（罗湖人民医院）
Standard plane recognition plays an important role in prenatal ultrasound (US) screening. Automatically recognizing the standard plane along with the corresponding anatomical structures in US image can not only facilitate US image interpretation but also improve diagnostic efficiency. In this study, we build a novel multi-label learning (MLL) scheme to identify multiple standard planes and corresponding anatomical structures of fetus simultaneously. Our contribution is three-fold. First, we represent the class correlation by word embeddings to capture the fine-grained semantic and latent stati…
Global-Graph Guided and Local-Graph Weighted Contrastive Learning for Unified Clustering on Incomplete and Noise Multi-View Data
Guangxi Normal University（广西师范学院）；University of Electronic Science and Technology of China（电子科技大学）；Singapore University of Technology and Design（新加坡科技设计大学）；Hainan University（海南大学）
BadCLIP: Dual-Embedding Guided Backdoor Attack on Multimodal Contrastive Learning
National University of Singapore（新加坡国立大学）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Beihang University（北京航空航天大学）；Sun Yat-sen University-Shenzhen（中山大学（深圳））
Diverse Normal Prototypes-Guided Contrastive Reconstruction for Medical Anomaly Detection
School of Mathematics, Shandong University（山东大学数学学院）；Department of Interventional Medicine and Minimally Invasive Oncology, The Second Qilu Hospital of Shandong University（山东大学第二齐鲁医院介入医学与微创肿瘤科）；Department of Neurosurgery, The Second Qilu Hospital of Shandong University（山东大学第二齐鲁医院神经外科）
In computer vision, image processing and computer graphics, image smoothing filtering is a very basic and important task and to be expected possessing good edge-preserving smoothing property. Here we address the problem that the edge-preserving ability of many popular local smoothing filters needs to be improved. In this paper, we propose the image Edge Restoring Filter (ERF) to restore the blur edge pixels in the output of local smoothing filters to be clear. The proposed filter can been implemented after many local smoothing filter (such as Box filter, Gaussian filter, Bilateral Filter, Guid…
Exemplar-based image translation establishes dense correspondences between a conditional input and an exemplar (from two different domains) for leveraging detailed exemplar styles to achieve realistic image translation. Existing work builds the cross-domain correspondences implicitly by minimizing feature-wise distances across the two domains. Without explicit exploitation of domain-invariant features, this approach may not reduce the domain gap effectively which often leads to sub-optimal correspondences and image translation. We design a Marginal Contrastive Learning Network (MCL-Net) that e…
Beijing University of Posts and Telecommunications（北京邮电大学）
Unsupervised person re-identification (Re-ID) is a promising and very challenging research problem in computer vision. Learning robust and discriminative features with unlabeled data is of central importance to Re-ID. Recently, more attention has been paid to unsupervised Re-ID algorithms based on clustered pseudo-label. However, the previous approaches did not fully exploit information of hard samples, simply using cluster centroid or all instances for contrastive learning. In this paper, we propose a Hard-sample Guided Hybrid Contrast Learning (HHCL) approach combining cluster-level loss wit…
LGE-Guided Cross-Modality Contrastive Learning for Gadolinium-Free Cardiomyopathy Screening in Cine CMR
School of Biomedical Engineering（生物医学工程学院）；State Key Laboratory of Advanced Medical Materials and Devices（先进医疗材料与设备国家重点实验室）；ShanghaiTech University（上海科技大学）；United Imaging Intelligence（联合影像智能）；Shanghai Clinical Research and Trial Center（上海临床研究与试验中心）；Capital Medical University（首都医科大学）；The First Hospital of Lanzhou University（兰州大学第一医院）
