The credibility and practicality of a reconstructed hand-object interaction sequence depend largely on its physical plausibility. However, due to high occlusions during hand-object interaction, physical plausibility remains a challenging criterion for purely vision-based tracking methods. To address this issue and enhance the results of existing hand trackers, this paper proposes a novel physically-aware hand motion de-noising method. Specifically, we introduce two learned loss terms that explicitly capture two crucial aspects of physical plausibility: grasp credibility and manipulation feasibility. These terms are used to train a physically-aware de-noising network. Qualitative and quantitative experiments demonstrate that our approach significantly improves both fine-grained physical plausibility and overall pose accuracy, surpassing current state-of-the-art de-noising methods.
We propose a fully-convolutional neural-network architecture for image denoising which is simple yet powerful. Its structure allows to exploit the gradual nature of the denoising process, in which shallow layers handle local noise statistics, while deeper layers recover edges and enhance textures. Our method advances the state-of-the-art when trained for different noise levels and distributions (both Gaussian and Poisson). In addition, we show that making the denoiser class-aware by exploiting semantic class information boosts performance, enhances textures and reduces artifacts.
The increasing demand for high image quality in mobile devices brings forth the need for better computational enhancement techniques, and image denoising in particular. At the same time, the images captured by these devices can be categorized into a small set of semantic classes. However simple, this observation has not been exploited in image denoising until now. In this paper, we demonstrate how the reconstruction quality improves when a denoiser is aware of the type of content in the image. To this end, we first propose a new fully convolutional deep neural network architecture which is sim…
This paper presents a novel non-local part-aware deep neural network to denoise point clouds by exploring the inherent non-local self-similarity in 3D objects and scenes. Different from existing works that explore small local patches, we design the non-local learning unit (NLU) customized with a graph attention module to adaptively capture non-local semantically-related features over the entire point cloud. To enhance the denoising performance, we cascade a series of NLUs to progressively distill the noise features from the noisy inputs. Further, besides the conventional surface reconstruction…
This paper presents a novel context-aware image denoising algorithm that combines an adaptive image smoothing technique and color reduction techniques to remove perturbation from adversarial images. Adaptive image smoothing is achieved using auto-threshold canny edge detection to produce an accurate edge map used to produce a blurred image that preserves more edge features. The proposed algorithm then uses color reduction techniques to reconstruct the image using only a few representative colors. Through this technique, the algorithm can reduce the effects of adversarial perturbations on image…
Texas A&M University（得克萨斯农工大学）；Nanyang Technological University（南洋理工大学）；The University of Sydney（悉尼大学）
Several recent works discussed application-driven image restoration neural networks, which are capable of not only removing noise in images but also preserving their semantic-aware details, making them suitable for various high-level computer vision tasks as the pre-processing step. However, such approaches require extra annotations for their high-level vision tasks, in order to train the joint pipeline using hybrid losses. The availability of those annotations is yet often limited to a few image sets, potentially restricting the general applicability of these methods to denoising more unseen …
Burst denoising is now more relevant than ever, as computational photography helps overcome sensitivity issues inherent in mobile phones and small cameras. A major challenge in burst-denoising is in coping with pixel misalignment, which was so far handled with rather simplistic assumptions of simple motion, or the ability to align in pre-processing. Such assumptions are not realistic in the presence of large motion and high levels of noise. We show that Neural Radiance Fields (NeRFs), originally suggested for physics-based novel-view rendering, can serve as a powerful framework for burst denoi…
Low Dose Computed Tomography (LDCT) has offered tremendous benefits in radiation restricted applications, but the quantum noise as resulted by the insufficient number of photons could potentially harm the diagnostic performance. Current image-based denoising methods tend to produce a blur effect on the final reconstructed results especially in high noise levels. In this paper, a deep learning based approach was proposed to mitigate this problem. An adversarially trained network and a sharpness detection network were trained to guide the training process. Experiments on both simulated and real …
SPADE: Spatial-Aware Denoising Network for Open-vocabulary Panoptic Scene Graph Generation with Long- and Local-range Context Reasoning
The Laboratory of Intelligent Collaborative Computing of UESTC（UESTC智能协同计算实验室）；Ubiquitous Intelligence and Trusted Services Key Laboratory of Sichuan Province（四川省 Ubiquitous Intelligence and Trusted Services 重点实验室）；Monash University（墨尔本大学）；Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ)（广东省人工智能与数字经济实验室（深圳））
Generative models have recently undergone significant advancement due to the diffusion models. The success of these models can be often attributed to their use of guidance techniques, such as classifier or classifier-free guidance, which provide effective mechanisms to trade-off between fidelity and diversity. However, these methods are not capable of guiding a generated image to be aware of its geometric configuration, e.g., depth, which hinders their application to areas that require a certain level of depth awareness. To address this limitation, we propose a novel guidance method for diffus…
Zhejiang University（浙江大学）；City University of Hong Kong（香港城市大学）
Although PIFu-based 3D human reconstruction methods are popular, the quality of recovered details is still unsatisfactory. In a sparse (e.g., 3 RGBD sensors) capture setting, the depth noise is typically amplified in the PIFu representation, resulting in flat facial surfaces and geometry-fallible bodies. In this paper, we propose a novel geometry-aware two-scale PIFu for 3D human reconstruction from sparse, noisy inputs. Our key idea is to exploit the complementary properties of depth denoising and 3D reconstruction, for learning a two-scale PIFu representation to reconstruct high-frequency fa…
BiEvLight: Bi-level Learning of Task-Aware Event Refinement for Low-Light Image Enhancement
College of Computer and Data Science, Fuzhou University（福州大学计算机与数据科学学院）；College of Computer Science and Technology, Shandong Technology and Business University（山东科技职业大学计算机科学与技术学院）
FDS: Frequency-Aware Denoising Score for Text-Guided Latent Diffusion Image Editing
School of Computer and Communication Sciences, EPFL（EPFL计算机与通信科学学院）；Department of Electrical and Photonics Engineering, DTU（丹麦技术大学电气与光子工程系）；Department of Electrical Engineering, Chalmers University of Technology（查尔姆斯理工大学电气工程系）
MPI-CBG/PKS (CSBD)（马克斯·普朗克分子细胞生物学与遗传学研究所/普朗克计算生物学中心（德累斯顿系统生物学中心））；Brno University of Technology（布尔诺理工大学）
Today, Convolutional Neural Networks (CNNs) are the leading method for image denoising. They are traditionally trained on pairs of images, which are often hard to obtain for practical applications. This motivates self-supervised training methods such as Noise2Void~(N2V) that operate on single noisy images. Self-supervised methods are, unfortunately, not competitive with models trained on image pairs. Here, we present 'Probabilistic Noise2Void' (PN2V), a method to train CNNs to predict per-pixel intensity distributions. Combining these with a suitable description of the noise, we obtain a compl…
Denoising-GS: Gaussian Splatting with Spatial-aware Denoising
School of Computer Science and Artificial Intelligence（计算机科学与人工智能学院）；Fudan University（复旦大学）；Software of Engineering（软件工程系）；Dalian University of Technology（大连理工大学）；Department of Information Engineering（信息工程系）；The Chinese University of Hong Kong（香港中文大学）；College of Computing and Data Science（计算与数据科学学院）；Nanyang Technological University（南洋理工大学）
ASCON: Anatomy-aware Supervised Contrastive Learning Framework for Low-dose CT Denoising
Institute of Science and Technology for Brain-Inspired Intelligence, Fudan University（复旦大学类脑智能科学与技术研究院）；School of Cyber Science and Engineering, Sichuan University（四川大学网络空间安全学院）
Although the advances of self-supervised blind denoising are significantly superior to conventional approaches without clean supervision in synthetic noise scenarios, it shows poor quality in real-world images due to spatially correlated noise corruption. Recently, pixel-shuffle downsampling (PD) has been proposed to eliminate the spatial correlation of noise. A study combining a blind spot network (BSN) and asymmetric PD (AP) successfully demonstrated that self-supervised blind denoising is applicable to real-world noisy images. However, PD-based inference may degrade texture details in the t…
Timestep-Aware Block Masking for Efficient Diffusion Model Inference
School of Computer Science, Wuhan University（武汉大学计算机学院）；School of Artificial Intelligence, Wuhan University（武汉大学人工智能学院）；School of Data Science, Fudan University（复旦大学数据科学学院）
Nanyang Technological University（南洋理工大学）；Alibaba Group（阿里巴巴集团）；Tianjin University（天津大学）；State Administration of Cultural Heritage（国家文物局）
Image denoising can remove natural noise that widely exists in images captured by multimedia devices due to low-quality imaging sensors, unstable image transmission processes, or low light conditions. Recent works also find that image denoising benefits the high-level vision tasks, e.g., image classification. In this work, we try to challenge this common sense and explore a totally new problem, i.e., whether the image denoising can be given the capability of fooling the state-of-the-art deep neural networks (DNNs) while enhancing the image quality. To this end, we initiate the very first attem…
Enhancing Video Physical Consistency via Role-aware Joint Training and Modality-decoupled Denoising
University of Science and Technology of China（中国科学技术大学）；University of Electronic Science and Technology of China（电子科技大学）；Fudan University（复旦大学）；Georgia Institute of Technology（佐治亚理工学院）；Shanghai Jiao Tong University（上海交通大学）
SharpXR: Structure-Aware Denoising for Pediatric Chest X-Rays
Federal University of Agriculture Abeokuta, Nigeria（尼日利亚阿贝奥库塔联邦农业大学）；University of Lagos, Nigeria（尼日利亚拉各斯大学）；University of Nigeria, Nsukka（尼日利亚恩苏克卡大学）；University of Benin, Nigeria（尼日利亚贝宁大学）；Olabisi Onabanjo University, Nigeria（尼日利亚奥拉比·奥纳班乔大学）；Federal University of Technology, Minna, Nigeria（尼日利亚米纳联邦技术大学）；McGill University, Canada（加拿大麦吉尔大学）；AGH University of Krakow, Poland（波兰克拉科夫AGH大学）
Center of Advanced Medical Computing and Analysis, MGH/HMS（麻省总医院/哈佛医学院高级医学计算与分析中心）；Johns Hopkins University（约翰斯·霍普金斯大学）；GE Healthcare（通用电气医疗）；University of Bern（伯尔尼大学）
In PET, the amount of relative (signal-dependent) noise present in different body regions can be significantly different and is inherently related to the number of counts present in that region. The number of counts in a region depends, in principle and among other factors, on the total administered activity, scanner sensitivity, image acquisition duration, radiopharmaceutical tracer uptake in the region, and patient local body morphometry surrounding the region. In theory, less amount of denoising operations is needed to denoise a high-count (low relative noise) image than images a low-count …
