We study image segmentation from an information-theoretic perspective, proposing a novel adversarial method that performs unsupervised segmentation by partitioning images into maximally independent sets. More specifically, we group image pixels into foreground and background, with the goal of minimizing predictability of one set from the other. An easily computed loss drives a greedy search process to maximize inpainting error over these partitions. Our method does not involve training deep networks, is computationally cheap, class-agnostic, and even applicable in isolation to a single unlabeled image. Experiments demonstrate that it achieves a new state-of-the-art in unsupervised segmentation quality, while being substantially faster and more general than competing approaches.
Czech Technical University（捷克技术大学）；Microsoft Research（微软研究院）；Center for Machine Perception（机器感知中心）
Most image labeling problems such as segmentation and image reconstruction are fundamentally ill-posed and suffer from ambiguities and noise. Higher order image priors encode high level structural dependencies between pixels and are key to overcoming these problems. However, these priors in general lead to computationally intractable models. This paper addresses the problem of discovering compact representations of higher order priors which allow efficient inference. We propose a framework for solving this problem which uses a recently proposed representation of higher order functions where th…
An Expectation-Maximization Algorithm for Training Clean Diffusion Models from Corrupted Observations
Academy for Advanced Interdisciplinary Studies, Peking University（北京大学前沿交叉学科研究院）；College of Future Technology, Peking University（北京大学未来技术学院）；National Biomedical Imaging Center, Peking University（北京大学国家生物医学成像中心）；Yuanpei College, Peking University（北京大学元培学院）；State Key Laboratory of Multimedia Information Processing, Peking University（北京大学多媒体信息处理国家重点实验室）
University of Bologna（博洛尼亚大学）；Massachusetts Institute of Technology（麻省理工学院）；Aprovechar Lab（Aprovechar实验室）
In this article, using an exemplar-based approach, we investigate the inpainting problem, introducing a new mathematical functional, whose minimization determines the quality of the reconstructions. The new functional expression takes into account of fnite differences terms, in a similar fashion to what happens in the theoretical Sobolev spaces. Moreover, we introduce a new priority index to determine the scanning order of the points to inpaint, prioritizing the uncertainty reduction in the choice. The achieved results highlight important theoretical-connected aspects of the inpainting by patc…
University of Stuttgart（斯图加特大学）；University of Houston（休斯顿大学）
Blind inpainting algorithms based on deep learning architectures have shown a remarkable performance in recent years, typically outperforming model-based methods both in terms of image quality and run time. However, neural network strategies typically lack a theoretical explanation, which contrasts with the well-understood theory underlying model-based methods. In this work, we leverage the advantages of both approaches by integrating theoretically founded concepts from transform domain methods and sparse approximations into a CNN-based approach for blind image inpainting. To this end, we pres…
University of Electronic Science and Technology of China（电子科技大学）；Henan University（河南大学）
Remote sensing image (RSI) inpainting plays an important role in real applications. Recently, fully-connected tensor network (FCTN) decomposition has been shown the remarkable ability to fully characterize the global correlation. Considering the global correlation and the nonlocal self-similarity (NSS) of RSIs, this paper introduces the FCTN decomposition to the whole RSI and its NSS groups, and proposes a novel nonlocal patch-based FCTN (NL-FCTN) decomposition for RSI inpainting. Different from other nonlocal patch-based methods, the NL-FCTN decomposition-based method, which increases tensor …
German Research Center for Artificial Intelligence (DFKI) GmbH（德国人工智能研究中心（DFKI）有限公司）；Saarland University（萨尔大学）
Exemplar-based inpainting is the process of reconstructing missing parts of an image by searching the remaining data for patches that fit seamlessly. The image is completed to a plausible-looking solution by repeatedly inserting the patch that is the best match according to some cost function. We present an acceleration structure that uses a multi-index scheme to accelerate this search procedure drastically, particularly in the case of very large datasets. The index scheme uses ideas such as dimensionality reduction and k-nearest neighbor search on space-filling curves that are well known in t…
Durham University（杜伦大学）；The Newcastle Upon Tyne Hospitals NHS Foundation Trust（泰恩河畔纽卡斯尔医院NHS基金会信托）；University of Glasgow（格拉斯哥大学）
A Cleft lip is a congenital abnormality requiring surgical repair by a specialist. The surgeon must have extensive experience and theoretical knowledge to perform surgery, and Artificial Intelligence (AI) method has been proposed to guide surgeons in improving surgical outcomes. If AI can be used to predict what a repaired cleft lip would look like, surgeons could use it as an adjunct to adjust their surgical technique and improve results. To explore the feasibility of this idea while protecting patient privacy, we propose a deep learning-based image inpainting method that is capable of coveri…
In this paper, we study color image inpainting as a pure quaternion matrix completion problem. In the literature, the theoretical guarantee for quaternion matrix completion is not well-established. Our main aim is to propose a new minimization problem with an objective combining nuclear norm and a quadratic loss weighted among three channels. To fill the theoretical vacancy, we obtain the error bound in both clean and corrupted regimes, which relies on some new results of quaternion matrices. A general Gaussian noise is considered in robust completion where all observations are corrupted. Moti…
AsyncDSB: Schedule-Asynchronous Diffusion Schrödinger Bridge for Image Inpainting
Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳））；ShenZhen SiFar Co., Ltd.（深圳思远有限公司）；Centre for Frontier AI Research（前沿人工智能研究中心）；Shengshu AI（圣树人工智能）
National University of Defense Technology（国防科技大学）；Harbin Institute of Technology（哈尔滨工业大学）；Dalian University of Technology（大连理工大学）
Nuclear norm maximization has shown the power to enhance the transferability of unsupervised domain adaptation model (UDA) in an empirical scheme. In this paper, we identify a new property termed equity, which indicates the balance degree of predicted classes, to demystify the efficacy of nuclear norm maximization for UDA theoretically. With this in mind, we offer a new discriminability-and-equity maximization paradigm built on squares loss, such that predictions are equalized explicitly. To verify its feasibility and flexibility, two new losses termed Class Weighted Squares Maximization (CWSM…
This paper addresses the issue of matching rigid 3D object points with 2D image points through point registration based on maximum likelihood principle in computer simulated images. Perspective projection is necessary when transforming 3D coordinate into 2D. The problem then recasts into a missing data framework where unknown correspondences are handled via mixture models. Adopting the Expectation Conditional Maximization for Point Registration (ECMPR), two different rotation and translation optimization algorithms are compared in this paper. We analyze in detail the associated consequences in…
Jiangsu Normal University（江苏师范大学）；Inner Mongolia University（内蒙古大学）；The University of Hong Kong（香港大学）；University of Electronic Science and Technology of China（电子科技大学）
The image nonlocal self-similarity (NSS) prior refers to the fact that a local patch often has many nonlocal similar patches to it across the image and has been widely applied in many recently proposed machining learning algorithms for image processing. However, there is no theoretical analysis on its working principle in the literature. In this paper, we discover a potential causality between NSS and low-rank property of color images, which is also available to grey images. A new patch group based NSS prior scheme is proposed to learn explicit NSS models of natural color images. The numerical…
We propose to use the concept of the Hamming bound to derive the optimal criteria for learning hash codes with a deep network. In particular, when the number of binary hash codes (typically the number of image categories) and code length are known, it is possible to derive an upper bound on the minimum Hamming distance between the hash codes. This upper bound can then be used to define the loss function for learning hash codes. By encouraging the margin (minimum Hamming distance) between the hash codes of different image categories to match the upper bound, we are able to learn theoretically o…
Explicit Mutual Information Maximization for Self-Supervised Learning
School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University（上海交通大学电子信息与电气工程学院）；Advanced Computing and Storage Laboratory, Huawei Technologies Co., Ltd.（华为技术有限公司先进计算与存储实验室）
