The denoising process of diffusion models can be interpreted as an approximate projection of noisy samples onto the data manifold. Moreover, the noise level in these samples approximates their distance to the underlying manifold. Building on this insight, we propose a novel method to enhance sample generation by aligning the estimated noise level with the true distance of noisy samples to the manifold. Specifically, we introduce a noise level correction network, leveraging a pre-trained denoising network, to refine noise level estimates during the denoising process. Additionally, we extend this approach to various image restoration tasks by integrating task-specific constraints, including inpainting, deblurring, super-resolution, colorization, and compressed sensing. Experimental results demonstrate that our method significantly improves sample quality in both unconstrained and constrained generation scenarios. Notably, the proposed noise level correction framework is compatible with existing denoising schedulers (e.g., DDIM), offering additional performance improvements.
Denoising diffusion probabilistic models (DDPMs) are a recent family of generative models that achieve state-of-the-art results. In order to obtain class-conditional generation, it was suggested to guide the diffusion process by gradients from a time-dependent classifier. While the idea is theoretically sound, deep learning-based classifiers are infamously susceptible to gradient-based adversarial attacks. Therefore, while traditional classifiers may achieve good accuracy scores, their gradients are possibly unreliable and might hinder the improvement of the generation results. Recent work dis…
We enhance the vanilla adversarial training method for unsupervised Automatic Speech Recognition (ASR) by a diffusion-GAN. Our model (1) injects instance noises of various intensities to the generator's output and unlabeled reference text which are sampled from pretrained phoneme language models with a length constraint, (2) asks diffusion timestep-dependent discriminators to separate them, and (3) back-propagates the gradients to update the generator. Word/phoneme error rate comparisons with wav2vec-U under Librispeech (3.1% for test-clean and 5.6% for test-other), TIMIT and MLS datasets, sho…
Enhancing Frequency Forgery Clues for Diffusion-Generated Image Detection
School of Computer and Communication Sciences, EPFL（瑞士联邦理工学院计算机与通信科学学院）；Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；University of Chinese Academy of Sciences（中国科学院大学）
Masked Conditional Diffusion Model for Enhancing Deepfake Detection
Chengdu University of Information Technology（成都信息工程大学）；Purdue University in Indianapolis（印第安纳波利斯普渡大学）；Johns Hopkins University（约翰斯·霍普金斯大学）；University at Albany, State University of New York (SUNY)（纽约州立大学奥尔巴尼分校）
Diffusion models can be used as learned priors for solving various inverse problems. However, most existing approaches are restricted to linear inverse problems, limiting their applicability to more general cases. In this paper, we build upon Denoising Diffusion Restoration Models (DDRM) and propose a method for solving some non-linear inverse problems. We leverage the pseudo-inverse operator used in DDRM and generalize this concept for other measurement operators, which allows us to use pre-trained unconditional diffusion models for applications such as JPEG artifact correction. We empiricall…
Noise Diffusion for Enhancing Semantic Faithfulness in Text-to-Image Synthesis
Beijing Normal University（北京师范大学）；University of Chinese Academy of Sciences（中国科学院大学）；University of Manchester（曼彻斯特大学）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Tsinghua University（清华大学）
Enhancing Privacy-Utility Trade-offs to Mitigate Memorization in Diffusion Models
School of Computer Science, Faculty of Engineering, The University of Sydney, Australia（悉尼大学计算机科学学院，工程学院）；School of Physics, Mathematics and Computing, The University of Western Australia, Australia（西澳大学物理、数学与计算学院）；Center for Research in Computer Vision, University of Central Florida, USA（佛罗里达中央大学计算机视觉研究中心）
Diffusion as Reasoning: Enhancing Object Navigation via Diffusion Model Conditioned on LLM-based Object-Room Knowledge
State Key Laboratory of Robotics and Systems, Harbin Institute of Technology, Harbin 150001, China（机器人系统国家重点实验室，哈尔滨工业大学，哈尔滨）；School of Mechanical and Aerospace Engineering, Nanyang Technological University, Singapore 639798, Singapore（机械与航空航天工程学院，南洋理工大学，新加坡）
SC-CDM: Enhancing Quality of Image Semantic Communication with a Compact Diffusion Model
School of Electronics and Information, Northwestern Polytechnical University（西北工业大学电子信息学院）；State Key Laboratory of Integrated Services Networks, Xidian University（西安电子科技大学综合业务网国家重点实验室）；Department of Electrical and Computer Engineering, University of Houston（休斯顿大学电气与计算机工程系）
