Illumination effects cause problems for many computer vision algorithms. We present a user-friendly interactive system for robust illumination-invariant image generation. Compared with the previous automated illumination-invariant image derivation approaches, our system enables users to specify a particular kind of illumination variation for removal. The derivation of illumination-invariant image is guided by the user input. The input is a stroke that defines an area covering a set of pixels whose intensities are influenced predominately by the illumination variation. This additional flexibility enhances the robustness for processing non-linearly rendered images and the images of the scenes where their illumination variations are difficult to estimate automatically. Finally, we present some evaluation results of our method.
Neural rendering algorithms introduce a fundamentally new approach for photorealistic rendering, typically by learning a neural representation of illumination on large numbers of ground truth images. When training for a given variable scene, i.e., changing objects, materials, lights and viewpoint, the space D of possible training data instances quickly becomes unmanageable as the dimensions of variable parameters increase. We introduce a novel Active Exploration method using Markov Chain Monte Carlo, which explores D, generating samples (i.e., ground truth renderings) that best help training a…
Illumination-insensitive Binary Descriptor for Visual Measurement Based on Local Inter-patch Invariance
University of Electronic Science and Technology of China (UESTC)（电子科技大学）；Sichuan University（四川大学）；institut superieur d’electronique de Paris - ISEP（巴黎高等电子学院（ISEP））
White balance (WB) is one of the first photo-finishing steps used to render a captured image to its final output. WB is applied to remove the color cast caused by the scene's illumination. Interactive photo-editing software allows users to manually select different regions in a photo as examples of the illumination for WB correction (e.g., clicking on achromatic objects). Such interactive editing is possible only with images saved in a RAW image format. This is because RAW images have no photo-rendering operations applied and photo-editing software is able to apply WB and other photo-finishing…
Institute for Computer Science（计算机科学研究所）；Cologne University of Applied Sciences（科隆应用技术大学）
This technical report describes an improved image mosaicking algorithm. It is based on Jain's logarithmic search algorithm [Jain 1981] which is coupled to the method of Kourogi (1999} for matching images in a video sequence. Logarithmic search has a better invariance against illumination changes than the original optical-flow-based method of Kourogi.
LightFuse: Relightable Interactive Gaussian Scene Reconstruction via Multi-Scan Fusion and 2D Gaussian Ray Tracing
ZJU-UIUC Institute, Zhejiang University（浙江大学ZJU-UIUC学院）；College of Mathematics, Sichuan University（四川大学数学学院）；College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；Chengdu Minto Tech（成都敏途科技）
We develop a linear algebraic framework for the shape-from-shading problem, because tensors arise when scalar (e.g. image) and vector (e.g. surface normal) fields are differentiated multiple times. The work is in two parts. In this first part we investigate when image derivatives exhibit invariance to changing illumination by calculating the statistics of image derivatives under general distributions on the light source. We computationally validate the hypothesis that image orientations (derivatives) provide increased invariance to illumination by showing (for a Lambertian model) that a shape-…
Taming the Light: Illumination-Invariant Semantic 3DGS-SLAM
School of Computer Science and Technology（计算机科学与技术学院）；Joint International Research Laboratory of Silk Road Multilingual Cognitive Computing（丝绸之路多语种认知计算联合国际实验室）；National Key Laboratory for Novel Software Technology（新型软件技术国家重点实验室）；Key Laboratory of Signal Detection and Processing（信号检测与处理重点实验室）
SEDDI（SEDDI 公司）；Universidad Carlos III de Madrid（马德里卡洛斯三世大学）；Universidad Rey Juan Carlos（胡安·卡洛斯国王大学）
We present a deep learning-based method for propagating spatially-varying visual material attributes (e.g. texture maps or image stylizations) to larger samples of the same or similar materials. For training, we leverage images of the material taken under multiple illuminations and a dedicated data augmentation policy, making the transfer robust to novel illumination conditions and affine deformations. Our model relies on a supervised image-to-image translation framework and is agnostic to the transferred domain; we showcase a semantic segmentation, a normal map, and a stylization. Following a…
Govt. Arts College, Melur（梅卢尔政府艺术学院）；Sri Meenakshi Govt. Arts College for Women, Madurai（马杜赖斯里米纳克希政府女子艺术学院）
Remote sensing image classification can be performed in many different ways to extract meaningful features. One common approach is to perform edge detection. A second approach is to try and detect whole shapes, given the fact that these shapes usually tend to have distinctive properties such as object foreground or background. To get optimal results, these two approaches can be combined. This paper adopts a combinatorial optimization method to adaptively select threshold based features to improve remote sensing image. Feature selection is an important combinatorial optimization problem in the …
Tongji University（同济大学）；Nanyang Technological University（南洋理工大学）；Meituan（美团）；City University of Hong Kong（香港城市大学）
Getting rid of the fundamental limitations in fitting to the paired training data, recent unsupervised low-light enhancement methods excel in adjusting illumination and contrast of images. However, for unsupervised low light enhancement, the remaining noise suppression issue due to the lacking of supervision of detailed signal largely impedes the wide deployment of these methods in real-world applications. Herein, we propose a novel Cycle-Interactive Generative Adversarial Network (CIGAN) for unsupervised low-light image enhancement, which is capable of not only better transferring illuminatio…
Neural Cone Radiosity for Interactive Global Illumination with Glossy Materials
College of Future Technology, Peking University（未来技术学院，北京大学）；School of Computer Science, Peking University（计算机学院，北京大学）；National Key Laboratory of Intelligent Parallel Technology（智能并行技术国家重点实验室）
Structured illumination microscopy (SIM) is a very important super-resolution microscopy technique, which provides high speed super-resolution with about two-fold spatial resolution enhancement. Several attempts aimed at improving the performance of SIM reconstruction algorithm have been reported. However, most of these highlight only one specific aspect of the SIM reconstruction -- such as the determination of the illumination pattern phase shift accurately -- whereas other key elements -- such as determination of modulation factor, estimation of object power spectrum, Wiener filtering freque…
We develop a linear algebraic framework for the shape-from-shading problem, because tensors arise when scalar (e.g. image) and vector (e.g. surface normal) fields are differentiated multiple times. Using this framework, we first investigate when image derivatives exhibit invariance to changing illumination by calculating the statistics of image derivatives under general distributions on the light source. Second, we apply that framework to develop Taylor-like expansions, and build a boot-strapping algorithm to find the polynomial surface solutions (under any light source) consistent with a give…
University of Science and Technology of China（中国科学技术大学）；Hefei University of Technology（合肥工业大学）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）
Person re-identification (Re-ID) in real-world scenarios usually suffers from various degradation factors, e.g., low-resolution, weak illumination, blurring and adverse weather. On the one hand, these degradations lead to severe discriminative information loss, which significantly obstructs identity representation learning; on the other hand, the feature mismatch problem caused by low-level visual variations greatly reduces retrieval performance. An intuitive solution to this problem is to utilize low-level image restoration methods to improve the image quality. However, existing restoration m…
Microsoft Mixed Reality and AI Zurich lab（微软混合现实与人工智能苏黎世实验室）；Department of Computer Science, ETH Zurich（苏黎世联邦理工学院计算机科学系）
To be invariant, or not to be invariant: that is the question formulated in this work about local descriptors. A limitation of current feature descriptors is the trade-off between generalization and discriminative power: more invariance means less informative descriptors. We propose to overcome this limitation with a disentanglement of invariance in local descriptors and with an online selection of the most appropriate invariance given the context. Our framework consists in a joint learning of multiple local descriptors with different levels of invariance and of meta descriptors encoding the r…
Google Inc.（谷歌公司）；USC Institute for Creative Technologies（南加州大学创新技术研究院）
We present a learning-based method to infer plausible high dynamic range (HDR), omnidirectional illumination given an unconstrained, low dynamic range (LDR) image from a mobile phone camera with a limited field of view (FOV). For training data, we collect videos of various reflective spheres placed within the camera's FOV, leaving most of the background unoccluded, leveraging that materials with diverse reflectance functions reveal different lighting cues in a single exposure. We train a deep neural network to regress from the LDR background image to HDR lighting by matching the LDR ground tru…
University of East Anglia（东英吉利大学）；University of Bath（巴斯大学）
A user-centric method for fast, interactive, robust and high-quality shadow removal is presented. Our algorithm can perform detection and removal in a range of difficult cases: such as highly textured and colored shadows. To perform detection an on-the-fly learning approach is adopted guided by two rough user inputs for the pixels of the shadow and the lit area. After detection, shadow removal is performed by registering the penumbra to a normalized frame which allows us efficient estimation of non-uniform shadow illumination changes, resulting in accurate and robust removal. Another major con…
Indian Institute of Technology Kanpur（印度理工学院坎普尔分校）
Hatching is a common method used by artists to accentuate the third dimension of a sketch, and to illuminate the scene. Our system SHAD3S attempts to compete with a human at hatching generic three-dimensional (3D) shapes, and also tries to assist her in a form exploration exercise. The novelty of our approach lies in the fact that we make no assumptions about the input other than that it represents a 3D shape, and yet, given a contextual information of illumination and texture, we synthesise an accurate hatch pattern over the sketch, without access to 3D or pseudo 3D. In the process, we contri…
Max Planck Institute for Informatics（马克斯·普朗克信息学研究所）；Technical University of Munich（慕尼黑工业大学）；Stanford University（斯坦福大学）
Editing of portrait images is a very popular and important research topic with a large variety of applications. For ease of use, control should be provided via a semantically meaningful parameterization that is akin to computer animation controls. The vast majority of existing techniques do not provide such intuitive and fine-grained control, or only enable coarse editing of a single isolated control parameter. Very recently, high-quality semantically controlled editing has been demonstrated, however only on synthetically created StyleGAN images. We present the first approach for embedding rea…
Tampere University（坦佩雷大学）；Aarhus University（奥胡斯大学）；INTEL（英特尔）
In this paper, we describe a new large dataset for illumination estimation. This dataset, called INTEL-TAU, contains 7022 images in total, which makes it the largest available high-resolution dataset for illumination estimation research. The variety of scenes captured using three different camera models, namely Canon 5DSR, Nikon D810, and Sony IMX135, makes the dataset appropriate for evaluating the camera and scene invariance of the different illumination estimation techniques. Privacy masking is done for sensitive information, e.g., faces. Thus, the dataset is coherent with the new General D…
The University of Texas at Dallas（德克萨斯大学达拉斯分校）；University of Maryland（马里兰大学）；University of Siegen（锡根大学）
Real-world face recognition requires an ability to perceive the unique features of an individual face across multiple, variable images. The primate visual system solves the problem of image invariance using cascades of neurons that convert images of faces into categorical representations of facial identity. Deep convolutional neural networks (DCNNs) also create generalizable face representations, but with cascades of simulated neurons. DCNN representations can be examined in a multidimensional "face space", with identities and image parameters quantified via their projections onto the axes tha…
Single-pixel imaging (SPI) is a novel optical imaging technique by replacing a two-dimensional pixelated sensor with a single-pixel detector and pattern illuminations. SPI have been extensively used for various tasks related to image acquisition and processing. In this work, a novel non-image-based task of playing Tic-Tac-Toe games interactively is merged into the framework of SPI. An optoelectronic artificial intelligent (AI) player with minimal digital computation can detect the game states, generate optimal moves and display output results mainly by pattern illumination and single-pixel det…
Recent studies show that deep neural networks are vulnerable to adversarial examples which can be generated via certain types of transformations. Being robust to a desired family of adversarial attacks is then equivalent to being invariant to a family of transformations. Learning invariant representations then naturally emerges as an important goal to achieve which we explore in this paper within specific application contexts. Specifically, we propose a cyclically-trained adversarial network to learn a mapping from image space to latent representation space and back such that the latent repres…
Tolerance to image variations (e.g. translation, scale, pose, illumination) is an important desired property of any object recognition system, be it human or machine. Moving towards increasingly bigger datasets has been trending in computer vision specially with the emergence of highly popular deep learning models. While being very useful for learning invariance to object inter- and intra-class shape variability, these large-scale wild datasets are not very useful for learning invariance to other parameters forcing researchers to resort to other tricks for training a model. In this work, we in…
Most popular hand-crafted key-point detectors such as Harris corner, SIFT, SURF aim to detect corners, blobs, junctions or other human defined structures in images. Though being robust with some geometric transformations, unintended scenarios or non-uniform lighting variations could significantly degrade their performance. Hence, a new detector that is flexible with context change and simultaneously robust with both geometric and non-uniform illumination variations is very desirable. In this paper, we propose a solution to this challenging problem by incorporating Scale and Rotation Invariant …
In this paper, we present a novel approach for object recognition in real-time by employing multilevel feature analysis and demonstrate the practicality of adapting feature extraction into a Naive Bayesian classification framework that enables simple, efficient, and robust performance. We also show the proposed method scales well as the number of level-classes grows. To effectively understand the patches surrounding a keypoint, the trained classifier uses hundreds of simple binary features and models class posterior probabilities. In addition, the classification process is computationally chea…
Universidad Nacional Autónoma de México（墨西哥国立自治大学）；Instituto de Óptica（光学研究所）
In many image processing applications, such as segmentation and classification, the selection of robust features descriptors is crucial to improve the discrimination capabilities in real world scenarios. In particular, it is well known that image textures constitute power visual cues for feature extraction and classification. In the past few years the local binary pattern (LBP) approach, a texture descriptor method proposed by Ojala et al., has gained increased acceptance due to its computational simplicity and more importantly for encoding a powerful signature for describing textures. However…
Hong Kong University of Science and Technology（香港科技大学）
This paper proposes a novel point-cloud-based place recognition system that adopts a deep learning approach for feature extraction. By using a convolutional neural network pre-trained on color images to extract features from a range image without fine-tuning on extra range images, significant improvement has been observed when compared to using hand-crafted features. The resulting system is illumination invariant, rotation invariant and robust against moving objects that are unrelated to the place identity. Apart from the system itself, we also bring to the community a new place recognition da…
Imperial College London（帝国理工学院）；University of Bern（伯尔尼大学）
Face analysis is a core part of computer vision, in which remarkable progress has been observed in the past decades. Current methods achieve recognition and tracking with invariance to fundamental modes of variation such as illumination, 3D pose, expressions. Notwithstanding, a much less standing mode of variation is motion deblurring, which however presents substantial challenges in face analysis. Recent approaches either make oversimplifying assumptions, e.g. in cases of joint optimization with other tasks, or fail to preserve the highly structured shape/identity information. Therefore, we p…
In this paper, we provide a novel dataset designed for camera invariant color constancy research. Camera invariance corresponds to the robustness of an algorithm's performance when run on images of the same scene taken by different cameras. Accordingly, images in the database correspond to several lab and field scenes each of which are captured by three different cameras with minimal registration errors. The lab scenes are also captured under five different illuminations. The spectral responses of cameras and the spectral power distributions of the lab light sources are also provided, as they …
Guangdong University of Technology（广东工业大学）；Southern University of Science and Technology（南方科技大学）
Visual place recognition (VPR) in condition-varying environments is still an open problem. Popular solutions are CNN-based image descriptors, which have been shown to outperform traditional image descriptors based on hand-crafted visual features. However, there are two drawbacks of current CNN-based descriptors: a) their high dimension and b) lack of generalization, leading to low efficiency and poor performance in applications. In this paper, we propose to use a convolutional autoencoder (CAE) to tackle this problem. We employ a high-level layer of a pre-trained CNN to generate features, and …
Jalisco Gov（哈利斯科州政府）；Universidad Autónoma de Guadalajara（瓜达拉哈拉自治大学）；Barcelona Tech（巴塞罗那理工大学）；Barcelona Supercomputing Center（巴塞罗那超级计算中心）；Centro de Investigaciones en Óptica（光学研究中心）
Convolutional Neural Networks (ConvNets) at present achieve remarkable performance in image classification tasks. However, current ConvNets cannot guarantee the capabilities of the mammalian visual systems such as invariance to contrast and illumination changes. Some ideas to overcome the illumination and contrast variations usually have to be tuned manually and tend to fail when tested with other types of data degradation. In this context, we present a new bio-inspired {entry} layer, M6, which detects low-level geometric features (lines, edges, and orientations) which are similar to patterns …
