Diffusion models have demonstrated exceptional ability in modeling complex image distributions, making them versatile plug-and-play priors for solving imaging inverse problems. However, their reliance on large-scale clean datasets for training limits their applicability in scenarios where acquiring clean data is costly or impractical. Recent approaches have attempted to learn diffusion models directly from corrupted measurements, but these methods either lack theoretical convergence guarantees or are restricted to specific types of data corruption. In this paper, we propose a principled expectation-maximization (EM) framework that iteratively learns diffusion models from noisy data with arbitrary corruption types. Our framework employs a plug-and-play Monte Carlo method to accurately estimate clean images from noisy measurements, followed by training the diffusion model using the reconstructed images. This process alternates between estimation and training until convergence. We evaluate the performance of our method across various imaging tasks, including inpainting, denoising, and deblurring. Experimental results demonstrate that our approach enables the learning of high-fidelity diffusion priors from noisy data, significantly enhancing reconstruction quality in imaging inverse problems.
University of Cambridge（剑桥大学）；Snap Inc.（Snap公司）；Singapore University of Technology and Design（新加坡科技设计大学）
While Generative Adversarial Networks (GANs) are fundamental to many generative modelling applications, they suffer from numerous issues. In this work, we propose a principled framework to simultaneously mitigate two fundamental issues in GANs: catastrophic forgetting of the discriminator and mode collapse of the generator. We achieve this by employing for GANs a contrastive learning and mutual information maximization approach, and perform extensive analyses to understand sources of improvements. Our approach significantly stabilizes GAN training and improves GAN performance for image synthes…
Heidelberg University（海德堡大学）；Technical University Dortmund（多特蒙德工业大学）
Image blur and image noise are imaging artifacts intrinsically arising in image acquisition. In this paper, we consider multi-frame blind deconvolution (MFBD), where image blur is described by the convolution of an unobservable, undeteriorated image and an unknown filter, and the objective is to recover the undeteriorated image from a sequence of its blurry and noisy observations. We present two new methods for MFBD, which, in contrast to previous work, do not require the estimation of the unknown filters. The first method is based on likelihood maximization and requires careful initialization…
Beijing Academy of Artificial Intelligence（北京智源人工智能研究院）；NLPR, Institute of Automation, CAS（中国科学院自动化研究所模式识别国家重点实验室）；Academy of Mathematics and Systems Science, CAS（中国科学院数学与系统科学研究院）；BAAI（北京智源人工智能研究院）；College of Artificial Intelligence, Hubei Business College（湖北商贸学院人工智能学院）；School of Mathematical Science, Beihang University（北京航空航天大学数学科学学院）；University of CAS（中国科学院大学）
Ellipsoid fitting is of general interest in machine vision, such as object detection and shape approximation. Most existing approaches rely on the least-squares fitting of quadrics, minimizing the algebraic or geometric distances, with additional constraints to enforce the quadric as an ellipsoid. However, they are susceptible to outliers and non-ellipsoid or biased results when the axis ratio exceeds certain thresholds. To address these problems, we propose a novel and robust method for ellipsoid fitting in a noisy, outlier-contaminated 3D environment. We explicitly model the ellipsoid by ker…
University of Fukui（福井大学）；Deakin University（迪肯大学）；Nanyang Technological University（南洋理工大学）
Complex blur such as the mixup of space-variant and space-invariant blur, which is hard to model mathematically, widely exists in real images. In this paper, we propose a novel image deblurring method that does not need to estimate blur kernels. We utilize a pair of images that can be easily acquired in low-light situations: (1) a blurred image taken with low shutter speed and low ISO noise; and (2) a noisy image captured with high shutter speed and high ISO noise. Slicing the blurred image into patches, we extend the Gaussian mixture model (GMM) to model the underlying intensity distribution …
Expectation-Maximization as the Engine of Scalable Medical Intelligence
Johns Hopkins University（约翰霍普金斯大学）；University of Bologna（博洛尼亚大学）；Italian Institute of Technology（意大利理工学院）；University Hospital Basel（巴塞尔大学医院）；DKFZ（德累斯顿生物医学研究中心）；Jagiellonian University（雅盖隆大学）；University of Illinois Urbana–Champaign（伊利诺伊大学厄巴纳-香槟分校）；University of California, San Francisco（旧金山大学）；NVIDIA（NVIDIA公司）；Johns Hopkins Medicine（约翰霍普金斯医学）
A Principled Self-Referenced Early Stopping Approach for Deep Image Prior
Department of Computational Mathematics, Science, & Engineering, Michigan State University（密歇根州立大学计算数学、科学与工程系）；Department of Electrical Engineering and Computer Science, University of Michigan（密歇根大学电气工程与计算机科学系）；X Computational Physics Division, Los Alamos National Laboratory（洛斯阿拉莫斯国家实验室计算物理部）；Michigan Institute for Computational Discovery & Engineering, University of Michigan（密歇根大学计算发现与工程研究所）；Mathematical Sciences, Michigan State University（密歇根州立大学数学科学系）
Institut Charles Delaunay (CNRS), Université de Technologie de Troyes（特鲁瓦技术大学，夏尔·德洛内研究所（法国国家科学研究中心））；School of Engineering and Physical Sciences, Heriot-Watt University（赫瑞瓦特大学，工程与物理科学学院）；LITIS lab, Université de Rouen（鲁昂大学，LITIS实验室）
In hyperspectral images, some spectral bands suffer from low signal-to-noise ratio due to noisy acquisition and atmospheric effects, thus requiring robust techniques for the unmixing problem. This paper presents a robust supervised spectral unmixing approach for hyperspectral images. The robustness is achieved by writing the unmixing problem as the maximization of the correntropy criterion subject to the most commonly used constraints. Two unmixing problems are derived: the first problem considers the fully-constrained unmixing, with both the non-negativity and sum-to-one constraints, while th…
A crucial step in single particle analysis (SPA) of cryogenic electron microscopy (Cryo-EM), 2D classification and alignment takes a collection of noisy particle images to infer orientations and group similar images together. Averaging these aligned and clustered noisy images produces a set of clean images, ready for further analysis such as 3D reconstruction. Fourier-Bessel steerable principal component analysis (FBsPCA) enables an efficient, adaptable, low-rank rotation operator. We extend the FBsPCA to additionally handle translations. In this extended FBsPCA representation, we use a probab…
Iterative Event-based Motion Segmentation by Variational Contrast Maximization
Keio University（庆应大学）；Technische Universität Berlin（柏林技术大学）；Einstein Center Digital Future（爱因斯坦数字未来研究中心）；Science of Intelligence Excellence Cluster, Germany（德国智能科学卓越集群）
The novel 2019 Coronavirus (COVID-19) infection has spread world widely and is currently a major healthcare challenge around the world. Chest Computed Tomography (CT) and X-ray images have been well recognized to be two effective techniques for clinical COVID-19 disease diagnoses. Due to faster imaging time and considerably lower cost than CT, detecting COVID-19 in chest X-ray (CXR) images is preferred for efficient diagnosis, assessment and treatment. However, considering the similarity between COVID-19 and pneumonia, CXR samples with deep features distributed near category boundaries are eas…
Clinical decision support using deep neural networks has become a topic of steadily growing interest. While recent work has repeatedly demonstrated that deep learning offers major advantages for medical image classification over traditional methods, clinicians are often hesitant to adopt the technology because its underlying decision-making process is considered to be intransparent and difficult to comprehend. In recent years, this has been addressed by a variety of approaches that have successfully contributed to providing deeper insight. Most notably, additive feature attribution methods are…
