Robustness is essential for deep neural networks, especially in security-sensitive applications. To this end, randomized smoothing provides theoretical guarantees for certifying robustness against adversarial perturbations. Recently, diffusion models have been successfully employed for randomized smoothing to purify noise-perturbed samples before making predictions with a standard classifier. While these methods excel at small perturbation radii, they struggle with larger perturbations and incur a significant computational overhead during inference compared to classical methods. To address this, we reformulate the generative modeling task along the diffusion trajectories in pixel space as a discriminative task in the latent space. Specifically, we use instance discrimination to achieve consistent representations along the trajectories by aligning temporally adjacent points. After fine-tuning based on the learned representations, our model enables implicit denoising-then-classification via a single prediction, substantially reducing inference costs. We conduct extensive experiments on various datasets and achieve state-of-the-art performance with minimal computation budget during inference. For example, our method outperforms the certified accuracy of diffusion-based methods on ImageNet across all perturbation radii by 5.3% on average, with up to 11.6% at larger radii, while reducing inference costs by 85$\times$ on average. Codes are available at: https://github.com/jiachenlei/rRCM.
Universidad Politécnica de Madrid（马德里理工大学）；Centre national de la recherche scientifique (CNRS)（法国国家科学研究中心）；Interdisciplinary Institute for Neuroscience (IINS)（跨学科神经科学研究所）；University of Bordeaux（波尔多大学）；Mechanobiology Institute（机械生物学研究所）；National University of Singapore（新加坡国立大学）
Microscopy images often suffer from high levels of noise, which can hinder further analysis and interpretation. Content-aware image restoration (CARE) methods have been proposed to address this issue, but they often require large amounts of training data and suffer from over-fitting. To overcome these challenges, we propose a novel framework for few-shot microscopy image denoising. Our approach combines a generative adversarial network (GAN) trained via contrastive learning (CL) with two structure preserving loss terms (Structural Similarity Index and Total Variation loss) to further improve t…
Conditional generative models of high-dimensional images have many applications, but supervision signals from conditions to images can be expensive to acquire. This paper describes Diffusion-Decoding models with Contrastive representations (D2C), a paradigm for training unconditional variational autoencoders (VAEs) for few-shot conditional image generation. D2C uses a learned diffusion-based prior over the latent representations to improve generation and contrastive self-supervised learning to improve representation quality. D2C can adapt to novel generation tasks conditioned on labels or mani…
Guangdong Cardiovascular Institute（广东省心血管病研究所）；Guangdong Academy of Medical Sciences（广东省医学科学院）；University of Notre Dame（圣母大学）
As one of the most commonly ordered imaging tests, computed tomography (CT) scan comes with inevitable radiation exposure that increases the cancer risk to patients. However, CT image quality is directly related to radiation dose, thus it is desirable to obtain high-quality CT images with as little dose as possible. CT image denoising tries to obtain high dose like high-quality CT images (domain X) from low dose low-quality CTimages (domain Y), which can be treated as an image-to-image translation task where the goal is to learn the transform between a source domain X (noisy images) and a targ…
Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院）
Recently, CycleGAN was shown to provide high-performance, ultra-fast denoising for low-dose X-ray computed tomography (CT) without the need for a paired training dataset. Although this was possible thanks to cycle consistency, CycleGAN requires two generators and two discriminators to enforce cycle consistency, demanding significant GPU resources and technical skills for training. A recent proposal of tunable CycleGAN with Adaptive Instance Normalization (AdaIN) alleviates the problem in part by using a single generator. However, two discriminators and an additional AdaIN code generator are st…
Pratt School of Engineering Duke University（普拉特工程学院 哥伦比亚大学）；Biomedical Imaging Center Rensselaer Polytechnic Institute（生物医学成像中心 莱文森理工学院）；Department of Computer Science University of Kentucky（计算机科学系 肯塔基大学）；Department of Medicine Department of Biomedical Informations University of Alabama at Birmingham（医学系 生物医学信息学系 亚拉巴马大学伯明翰分校）；School of Engineering Sciences Department of Physics KTH Royal Institute of Technology（工程科学学院 物理系 瑞典皇家理工学院）
FoundDiff: Foundational Diffusion Model for Generalizable Low-Dose CT Denoising
Institute of Science and Technology for Brain-inspired Intelligence and MOE Frontiers Center for Brain Science, Fudan University（脑启发智能科学技术研究所和脑科学前沿中心，复旦大学）；Shanghai Key Lab of Intelligent Information Processing, School of Computer Science, Fudan University（智能信息处理上海市重点实验室，复旦大学计算机科学学院）；School of Cyber Science and Engineering, Sichuan University（四川大学网络科学与工程学院）；School of Biomedical Engineering, Shanghai Jiao Tong University（上海交通大学生物医学工程学院）
Department of Engineering Science, University of Oxford, Oxford, United Kingdom（牛津大学工程科学系）；School of Computer Science, Carleton University, Ottawa, Canada（卡尔顿大学计算机科学学院）
Enhancing Video Physical Consistency via Role-aware Joint Training and Modality-decoupled Denoising
University of Science and Technology of China（中国科学技术大学）；University of Electronic Science and Technology of China（电子科技大学）；Fudan University（复旦大学）；Georgia Institute of Technology（佐治亚理工学院）；Shanghai Jiao Tong University（上海交通大学）
AnF-DiffPET: Anatomy- and Frequency-Guided Diffusion for PET/CT Denoising
College of Medicine and Biological Information Engineering, Northeastern University（东北大学医学与生物信息工程学院）；Tohoku University School of Medicine（东北大学医学院）；Department of Clinical Neurosciences, University of Cambridge（剑桥大学临床神经科学系）
Conditional Denoising Diffusion Model-Based Robust MR Image Reconstruction from Highly Undersampled Data
Department of Mathematics and Systems Engineering（数学与系统工程系）；Department of Biomedical Engineering & Science（生物医学工程与科学系）；Florida Institute of Technology（佛罗里达理工学院）；A. A. Martinos Center for Biomedical Imaging（A.A. Martinos生物医学成像中心）；Massachusetts General Hospital/Harvard Medical School（麻省总医院/哈佛医学院）；Department of Radiology（放射科）；Embry-Riddle Aeronautical University（艾姆伯-里尔航空大学）
Illinois Institute of Technology（伊利诺伊理工学院）；Wuhan University（武汉大学）；Snap Inc.（Snap公司）
Diffusion probabilistic models (DPMs) have become a popular approach to conditional generation, due to their promising results and support for cross-modal synthesis. A key desideratum in conditional synthesis is to achieve high correspondence between the conditioning input and generated output. Most existing methods learn such relationships implicitly, by incorporating the prior into the variational lower bound. In this work, we take a different route -- we explicitly enhance input-output connections by maximizing their mutual information. To this end, we introduce a Conditional Discrete Contr…
ASCON: Anatomy-aware Supervised Contrastive Learning Framework for Low-dose CT Denoising
Institute of Science and Technology for Brain-Inspired Intelligence, Fudan University（复旦大学类脑智能科学与技术研究院）；School of Cyber Science and Engineering, Sichuan University（四川大学网络空间安全学院）
Pyramid Self-Contrastive Learning for Single-shot Test-time Ultrasound Image Denoising
Department of Electrical and Computer Engineering, The University of Hong Kong（香港大学电子与计算机工程系）；Department of Biomedical Engineering, Duke University（达特茅斯大学生物医学工程系）
Adversarial Robustness Distillation (ARD) is a novel method to boost the robustness of small models. Unlike general adversarial training, its robust knowledge transfer can be less easily restricted by the model capacity. However, the teacher model that provides the robustness of knowledge does not always make correct predictions, interfering with the student's robust performances. Besides, in the previous ARD methods, the robustness comes entirely from one-to-one imitation, ignoring the relationship between examples. To this end, we propose a novel structured ARD method called Contrastive Rela…
Sun Yat-sen University（中山大学）；The Key Laboratory of Machine Intelligence and Advanced Computing (Sun Yat-sen University), Ministry of Education（机器智能与先进计算教育部重点实验室（中山大学））；JD AI Research（京东人工智能研究院）
BERT-type structure has led to the revolution of vision-language pre-training and the achievement of state-of-the-art results on numerous vision-language downstream tasks. Existing solutions dominantly capitalize on the multi-modal inputs with mask tokens to trigger mask-based proxy pre-training tasks (e.g., masked language modeling and masked object/frame prediction). In this work, we argue that such masked inputs would inevitably introduce noise for cross-modal matching proxy task, and thus leave the inherent vision-language association under-explored. As an alternative, we derive a particul…
Jilin University（吉林大学）；Baidu Research（百度研究院）；National Engineering Laboratory for Deep Learning Technology and Application（深度学习技术及应用国家工程实验室）
Current semi-supervised semantic segmentation methods mainly focus on designing pixel-level consistency and contrastive regularization. However, pixel-level regularization is sensitive to noise from pixels with incorrect predictions, and pixel-level contrastive regularization has memory and computational cost with O(pixel_num^2). To address the issues, we propose a novel region-level contrastive and consistency learning framework (RC^2L) for semi-supervised semantic segmentation. Specifically, we first propose a Region Mask Contrastive (RMC) loss and a Region Feature Contrastive (RFC) loss to …
This paper introduces SelfMatch, a semi-supervised learning method that combines the power of contrastive self-supervised learning and consistency regularization. SelfMatch consists of two stages: (1) self-supervised pre-training based on contrastive learning and (2) semi-supervised fine-tuning based on augmentation consistency regularization. We empirically demonstrate that SelfMatch achieves the state-of-the-art results on standard benchmark datasets such as CIFAR-10 and SVHN. For example, for CIFAR-10 with 40 labeled examples, SelfMatch achieves 93.19% accuracy that outperforms the strong p…
ConsistencyDet: A Few-step Denoising Framework for Object Detection Using the Consistency Model
College of Computer Science and Engineering, Shandong University of Science and Technology（山东科技大学计算机科学与工程学院）；Shenzhen Research Institute of Big Data（深圳大数据研究院）；Zhejiang Normal University（浙江师范大学）；Chongqing University of Posts and Telecommunications（重庆邮电大学）；Nanjing University of Science and Technology（南京理工大学）
ARC Lab（ARC实验室）；Peking University（北京大学）；Huazhong University of Science and Technology（华中科技大学）；Foundation Technology Center（基础技术中心）；Tsinghua University（清华大学）；Tencent PCG（腾讯平台与内容事业群）
Since the development of self-supervised visual representation learning from contrastive learning to masked image modeling (MIM), there is no significant difference in essence, that is, how to design proper pretext tasks for vision dictionary look-up. MIM recently dominates this line of research with state-of-the-art performance on vision Transformers (ViTs), where the core is to enhance the patch-level visual context capturing of the network via denoising auto-encoding mechanism. Rather than tailoring image tokenizers with extra training stages as in previous works, we unleash the great poten…
