Color constancy is our ability to perceive constant colors across varying illuminations. Here, we trained deep neural networks to be color constant and evaluated their performance with varying cues. Inputs to the networks consisted of the cone excitations in 3D-rendered images of 2115 different 3D-shapes, with spectral reflectances of 1600 different Munsell chips, illuminated under 278 different natural illuminations. The models were trained to classify the reflectance of the objects. One network, Deep65, was trained under a fixed daylight D65 illumination, while DeepCC was trained under varying illuminations. Testing was done with 4 new illuminations with equally spaced CIEL*a*b* chromaticities, 2 along the daylight locus and 2 orthogonal to it. We found a high degree of color constancy for DeepCC, and constancy was higher along the daylight locus. When gradually removing cues from the scene, constancy decreased. High levels of color constancy were achieved with different DNN architectures. Both ResNets and classical ConvNets of varying degrees of complexity performed well. However, DeepCC, a convolutional network, represented colors along the 3 color dimensions of human color vision, while ResNets showed a more complex representation.
University of Amsterdam（阿姆斯特丹大学）；DUniversum（3DUniversum）
In this paper, the aim is multi-illumination color constancy. However, most of the existing color constancy methods are designed for single light sources. Furthermore, datasets for learning multiple illumination color constancy are largely missing. We propose a seed (physics driven) based multi-illumination color constancy method. GANs are exploited to model the illumination estimation problem as an image-to-image domain translation problem. Additionally, a novel multi-illumination data augmentation method is proposed. Experiments on single and multi-illumination datasets show that our methods…
Color constancy is the problem of inferring the color of the light that illuminated a scene, usually so that the illumination color can be removed. Because this problem is underconstrained, it is often solved by modeling the statistical regularities of the colors of natural objects and illumination. In contrast, in this paper we reformulate the problem of color constancy as a 2D spatial localization task in a log-chrominance space, thereby allowing us to apply techniques from object detection and structured prediction to the color constancy problem. By directly learning how to discriminate bet…
Tampere University（坦佩雷大学）；Huawei Tampere Research Center（华为坦佩雷研究中心）；Czech Technical University in Prague（布拉格捷克理工大学）
Temporal Color Constancy (CC) is a recently proposed approach that challenges the conventional single-frame color constancy. The conventional approach is to use a single frame - shot frame - to estimate the scene illumination color. In temporal CC, multiple frames from the view finder sequence are used to estimate the color. However, there are no realistic large scale temporal color constancy datasets for method evaluation. In this work, a new temporal CC benchmark is introduced. The benchmark comprises of (1) 600 real-world sequences recorded with a high-resolution mobile phone camera, (2) a …
Most digital camera pipelines use color constancy methods to reduce the influence of illumination and camera sensor on the colors of scene objects. The highest accuracy of color correction is obtained with learning-based color constancy methods, but they require a significant amount of calibrated training images with known ground-truth illumination. Such calibration is time consuming, preferably done for each sensor individually, and therefore a major bottleneck in acquiring high color constancy accuracy. Statistics-based methods do not require calibrated training images, but they are less acc…
Bio-inspired Color Constancy: From Gray Anchoring Theory to Gray Pixel Methods
MOE Key Laboratory for Neuroinformation, School of Life Science and Technology, University of Electronic Science and Technology of China（教育部神经信息学重点实验室，生命科学与技术学院，电子科学与技术大学）；School of Electronic Engineering and Automation, Guilin University of Electronic Technology（电子工程与自动化学院，桂林电子科技大学）
Tampere University（坦佩雷大学）；Aarhus University（奥胡斯大学）；Aristotle University of Thessaloniki（塞萨洛尼基亚里士多德大学）；INTEL（英特尔）
In this paper, we propose a novel color constancy approach, called Bag of Color Features (BoCF), building upon Bag-of-Features pooling. The proposed method substantially reduces the number of parameters needed for illumination estimation. At the same time, the proposed method is consistent with the color constancy assumption stating that global spatial information is not relevant for illumination estimation and local information ( edges, etc.) is sufficient. Furthermore, BoCF is consistent with color constancy statistical approaches and can be interpreted as a learning-based generalization of …
Faculty of Electrical Engineering and Computing（电气与计算机工程学院）；University of Zagreb（萨格勒布大学）
Implementing color constancy as a pre-processing step in contemporary digital cameras is of significant importance as it removes the influence of scene illumination on object colors. Several benchmark color constancy datasets have been created for the purpose of developing and testing new color constancy methods. However, they all have numerous drawbacks including a small number of images, erroneously extracted ground-truth illuminations, long histories of misuses, violations of their stated assumptions, etc. To overcome such and similar problems, in this paper a color constancy benchmark data…
Computational color constancy refers to the problem of computing the illuminant color so that the images of a scene under varying illumination can be normalized to an image under the canonical illumination. In this paper, we adopt a deep learning framework for the illumination estimation problem. The proposed method works under the assumption of uniform illumination over the scene and aims for the accurate illuminant color computation. Specifically, we trained the convolutional neural network to solve the problem by casting the color constancy problem as an illumination classification problem.…
We present Fast Fourier Color Constancy (FFCC), a color constancy algorithm which solves illuminant estimation by reducing it to a spatial localization task on a torus. By operating in the frequency domain, FFCC produces lower error rates than the previous state-of-the-art by 13-20% while being 250-3000 times faster. This unconventional approach introduces challenges regarding aliasing, directional statistics, and preconditioning, which we address. By producing a complete posterior distribution over illuminants instead of a single illuminant estimate, FFCC enables better training techniques, a…
Tampere University（坦佩雷大学）；Aarhus University（奥胡斯大学）；Intel Corporation（英特尔公司）
In this paper, we study the importance of pre-training for the generalization capability in the color constancy problem. We propose two novel approaches based on convolutional autoencoders: an unsupervised pre-training algorithm using a fine-tuned encoder and a semi-supervised pre-training algorithm using a novel composite-loss function. This enables us to solve the data scarcity problem and achieve competitive, to the state-of-the-art, results while requiring much fewer parameters on ColorChecker RECommended dataset. We further study the over-fitting phenomenon on the recently introduced vers…
We present "Cross-Camera Convolutional Color Constancy" (C5), a learning-based method, trained on images from multiple cameras, that accurately estimates a scene's illuminant color from raw images captured by a new camera previously unseen during training. C5 is a hypernetwork-like extension of the convolutional color constancy (CCC) approach: C5 learns to generate the weights of a CCC model that is then evaluated on the input image, with the CCC weights dynamically adapted to different input content. Unlike prior cross-camera color constancy models, which are usually designed to be agnostic t…
In this paper, we propose a novel multi-color balance adjustment for color constancy. The proposed method, called "n-color balancing," allows us not only to perfectly correct n target colors on the basis of corresponding ground truth colors but also to correct colors other than the n colors. In contrast, although white-balancing can perfectly adjust white, colors other than white are not considered in the framework of white-balancing in general. In an experiment, the proposed multi-color balancing is demonstrated to outperform both conventional white and multi-color balance adjustments includi…
Xidian University（西安电子科技大学）；Hong Kong University of Science and Technology（香港科技大学）
Information about the illuminant color is well contained in both achromatic regions and the specular components of highlight regions. In this paper, we propose a novel way to achieve color constancy by exploiting such clues. The key to our approach lies in the use of suitably extracted derivative colors, which are able to compute the illuminant color robustly with kernel density estimation. While extracting derivative colors from achromatic regions to approximate the illuminant color well is basically straightforward, the success of our extraction in highlight regions is attributed to the diff…
The goal of computational color constancy is to preserve the perceptive colors of objects under different lighting conditions by removing the effect of color casts caused by the scene's illumination. With the rapid development of deep learning based techniques, significant progress has been made in image semantic segmentation. In this work, we exploit the semantic information together with the color and spatial information of the input image in order to remove color casts. We train a convolutional neural network (CNN) model that learns to estimate the illuminant color and gamma correction para…
In this paper, we provide a novel dataset designed for camera invariant color constancy research. Camera invariance corresponds to the robustness of an algorithm's performance when run on images of the same scene taken by different cameras. Accordingly, images in the database correspond to several lab and field scenes each of which are captured by three different cameras with minimal registration errors. The lab scenes are also captured under five different illuminations. The spectral responses of cameras and the spectral power distributions of the lab light sources are also provided, as they …
Tampere University of Technology（坦佩雷理工大学）；Intel Finland（英特尔芬兰公司）；Czech Technical University（捷克理工大学）
Computational color constancy that requires esti- mation of illuminant colors of images is a fundamental yet active problem in computer vision, which can be formulated into a regression problem. To learn a robust regressor for color constancy, obtaining meaningful imagery features and capturing latent correlations across output variables play a vital role. In this work, we introduce a novel deep structured-output regression learning framework to achieve both goals simultaneously. By borrowing the power of deep convolutional neural networks (CNN) originally designed for visual recognition, the …
Norwegian Colour and Visual Computing Laboratory（挪威色彩与视觉计算实验室）；NTNU（挪威科技大学）
Color Constancy is the ability of the human visual system to perceive colors unchanged independently of the illumination. Giving a machine this feature will be beneficial in many fields where chromatic information is used. Particularly, it significantly improves scene understanding and object recognition. In this paper, we propose transfer learning-based algorithm, which has two main features: accuracy higher than many state-of-the-art algorithms and simplicity of implementation. Despite the fact that GoogLeNet was used in the experiments, given approach may be applied to any CNN. Additionally…
There is active research targeting local image manipulations that can fool deep neural networks (DNNs) into producing incorrect results. This paper examines a type of global image manipulation that can produce similar adverse effects. Specifically, we explore how strong color casts caused by incorrectly applied computational color constancy - referred to as white balance (WB) in photography - negatively impact the performance of DNNs targeting image segmentation and classification. In addition, we discuss how existing image augmentation methods used to improve the robustness of DNNs are not we…
In this study, a novel illuminant color estimation framework is proposed for computational color constancy, which incorporates the high representational capacity of deep-learning-based models and the great interpretability of assumption-based models. The well-designed building block, feature map reweight unit (ReWU), helps to achieve comparative accuracy on benchmark datasets with respect to prior state-of-the-art deep learning based models while requiring more compact model size and cheaper computational cost. In addition to local color estimation, a confidence estimation branch is also inclu…
Fudan University（复旦大学）；Northeastern State University（东北州立大学）；Nokia-Bell Labs（诺基亚贝尔实验室）
As a novel method eliminating chromatic aberration on objects, computational color constancy has becoming a fundamental prerequisite for many computer vision applications. Among algorithms performing this task, the learning-based ones have achieved great success in recent years. However, they fail to fully consider the spatial information of images, leaving plenty of room for improvement of the accuracy of illuminant estimation. In this paper, by exploiting the spatial information of images, we propose a color constancy algorithm called Attention Dense Color Constancy (ADCC) using convolutiona…
Computational color constancy is a very important topic in computer vision and has attracted many researchers' attention. Recently, lots of research has shown the effects of high level visual content information for illumination estimation. However, all of these existing methods are essentially combinational strategies in which image's content analysis is only used to guide the combination or selection from a variety of individual illumination estimation methods. In this paper, we propose a novel bilayer sparse coding model for illumination estimation that considers image similarity in terms o…
Tampere University（坦佩雷大学）；South China University of Technology（华南理工大学）
We briefly introduce two submissions to the Illumination Estimation Challenge, in the Int'l Workshop on Color Vision, affiliated to the 11th Int'l Symposium on Image and Signal Processing and Analysis. The Fourier-transform-based submission is ranked 3rd, and the statistical Gray-pixel-based one ranked 6th.
South China University of Technology（华南理工大学）；Tampere University（坦佩雷大学）；Chinese Academy of Sciences（中国科学院）
Regressing the illumination of a scene from the representations of object appearances is popularly adopted in computational color constancy. However, it's still challenging due to intrinsic appearance and label ambiguities caused by unknown illuminants, diverse reflection property of materials and extrinsic imaging factors (such as different camera sensors). In this paper, we introduce a novel algorithm by Cascading Convolutional Color Constancy (in short, C4) to improve robustness of regression learning and achieve stable generalization capability across datasets (different cameras and scenes…
Toyota Technological Institute at Chicago（芝加哥丰田技术研究院）
Color constancy is the recovery of true surface color from observed color, and requires estimating the chromaticity of scene illumination to correct for the bias it induces. In this paper, we show that the per-pixel color statistics of natural scenes---without any spatial or semantic context---can by themselves be a powerful cue for color constancy. Specifically, we describe an illuminant estimation method that is built around a "classifier" for identifying the true chromaticity of a pixel given its luminance (absolute brightness across color channels). During inference, each pixel's observed …
In this paper, we present CLCC, a novel contrastive learning framework for color constancy. Contrastive learning has been applied for learning high-quality visual representations for image classification. One key aspect to yield useful representations for image classification is to design illuminant invariant augmentations. However, the illuminant invariant assumption conflicts with the nature of the color constancy task, which aims to estimate the illuminant given a raw image. Therefore, we construct effective contrastive pairs for learning better illuminant-dependent features via a novel raw…
The Norwegian Colour and Visual Computing Laboratory, NTNU（挪威色彩与视觉计算实验室，挪威科技大学）
Non-uniform and multi-illuminant color constancy are important tasks, the solution of which will allow to discard information about lighting conditions in the image. Non-uniform illumination and shadows distort colors of real-world objects and mostly do not contain valuable information. Thus, many computer vision and image processing techniques would benefit from automatic discarding of this information at the pre-processing step. In this work we propose novel view on this classical problem via generative end-to-end algorithm based on image conditioned Generative Adversarial Network. We also d…
In this paper, a learning-free color constancy algorithm called the Patch-wise Bright Pixels (PBP) is proposed. In this algorithm, an input image is first downsampled and then cut equally into a few patches. After that, according to the modified brightness of each patch, a proper fraction of brightest pixels in the patch is selected. Finally, Gray World (GW)-based methods are applied to the selected bright pixels to estimate the illuminant of the scene. Experiments on NUS $8$-Camera Dataset show that the PBP algorithm outperforms the state-of-the-art learning-free methods as well as a broad ra…
Tampere University（坦佩雷大学）；Finnish Environment Institute（芬兰环境研究所）；Aarhus University（奥胡斯大学）；Xiaomi（小米）
In this paper, we propose a novel unsupervised color constancy method, called Probabilistic Color Constancy (PCC). We define a framework for estimating the illumination of a scene by weighting the contribution of different image regions using a graph-based representation of the image. To estimate the weight of each (super-)pixel, we rely on two assumptions: (Super-)pixels with similar colors contribute similarly and darker (super-)pixels contribute less. The resulting system has one global optimum solution. The proposed method achieves competitive performance, compared to the state-of-the-art,…
Huawei Noah’s Ark Lab（华为诺亚方舟实验室）；Mila（米拉）；Technical University of Munich（慕尼黑工业大学）
Contemporary approaches frame the color constancy problem as learning camera specific illuminant mappings. While high accuracy can be achieved on camera specific data, these models depend on camera spectral sensitivity and typically exhibit poor generalisation to new devices. Additionally, regression methods produce point estimates that do not explicitly account for potential ambiguities among plausible illuminant solutions, due to the ill-posed nature of the problem. We propose a Bayesian framework that naturally handles color constancy ambiguity via a multi-hypothesis strategy. Firstly, we s…
University of Milan-Bicocca（米兰比可卡大学）；University of Pavia（帕维亚大学）
In this work we describe a Convolutional Neural Network (CNN) to accurately predict the scene illumination. Taking image patches as input, the CNN works in the spatial domain without using hand-crafted features that are employed by most previous methods. The network consists of one convolutional layer with max pooling, one fully connected layer and three output nodes. Within the network structure, feature learning and regression are integrated into one optimization process, which leads to a more effective model for estimating scene illumination. This approach achieves state-of-the-art performa…
Faculty of Computer Science & Information Systems, Universiti Teknologi Malaysia（马来西亚理工大学计算机科学与信息系统学院）；Centre of Real Estate Studies, Universiti Teknologi Malaysia（马来西亚理工大学房地产研究中心）
This paper present our color constancy investigation in the hybridization of Wireless LAN and Camera positioning in the mobile phone. Five typical color constancy schemes are analyzed in different location environment. The results can be used to combine with RF signals from Wireless LAN positioning by using model fitting approach in order to establish absolute positioning output. There is no conventional searching algorithm required, thus it is expected to reduce the complexity of computation. Finally we present our preliminary results to illustrate the indoor positioning algorithm performance…
Tencent AI Lab（腾讯人工智能实验室）；University of Macau（澳门大学）；University of Science and Technology of China（中国科学技术大学）；Noak’s Ark Lab, Huawei Technologies（华为技术有限公司诺亚方舟实验室）；Dalian University of Technology（大连理工大学）
Color constancy aims to restore the constant colors of a scene under different illuminants. However, due to the existence of camera spectral sensitivity, the network trained on a certain sensor, cannot work well on others. Also, since the training datasets are collected in certain environments, the diversity of illuminants is limited for complex real world prediction. In this paper, we tackle these problems via two aspects. First, we propose cross-sensor self-supervised training to train the network. In detail, we consider both the general sRGB images and the white-balanced RAW images from cur…
University of Electronic Science and Technology of China（电子科技大学）；Chinese Academy of Sciences（中国科学院）
It is an ill-posed problem to recover the true scene colors from a color biased image by discounting the effects of scene illuminant and camera spectral sensitivity (CSS) at the same time. Most color constancy (CC) models have been designed to first estimate the illuminant color, which is then removed from the color biased image to obtain an image taken under white light, without the explicit consideration of CSS effect on CC. This paper first studies the CSS effect on illuminant estimation arising in the inter-dataset-based CC (inter-CC), i.e., training a CC model on one dataset and then test…
Tampere University（坦佩雷大学）；Aarhus University（奥胡斯大学）；INTEL（英特尔）
In this paper, we describe a new large dataset for illumination estimation. This dataset, called INTEL-TAU, contains 7022 images in total, which makes it the largest available high-resolution dataset for illumination estimation research. The variety of scenes captured using three different camera models, namely Canon 5DSR, Nikon D810, and Sony IMX135, makes the dataset appropriate for evaluating the camera and scene invariance of the different illumination estimation techniques. Privacy masking is done for sensitive information, e.g., faces. Thus, the dataset is coherent with the new General D…
Digital camera pipelines employ color constancy methods to estimate an unknown scene illuminant, in order to re-illuminate images as if they were acquired under an achromatic light source. Fully-supervised learning approaches exhibit state-of-the-art estimation accuracy with camera-specific labelled training imagery. Resulting models typically suffer from domain gaps and fail to generalise across imaging devices. In this work, we propose a new approach that affords fast adaptation to previously unseen cameras, and robustness to changes in capture device by leveraging annotated samples across d…
Shahrekord University（沙赫雷克德大学）；University of Groningen（格罗宁根大学）
Despite the recent success of state-of-the-art 3D object recognition approaches, service robots are frequently failed to recognize many objects in real human-centric environments. For these robots, object recognition is a challenging task due to the high demand for accurate and real-time response under changing and unpredictable environmental conditions. Most of the recent approaches use either the shape information only and ignore the role of color information or vice versa. Furthermore, they mainly utilize the $L_n$ Minkowski family functions to measure the similarity of two object views, wh…
Boosting Illuminant Estimation in Deep Color Constancy through Enhancing Brightness Robustness
School of Computer Science and Cyber Engineering, Guangzhou University（广州大学计算机科学与网络工程学院）；State Key Laboratory of Blockchain and Data Security, Zhejiang University（浙江大学区块链与数据安全国家重点实验室）；Information Security Research Group, University College London（伦敦大学学院信息安全研究组）
Primary visual cortex contributes to color constancy by predicting rather than discounting the illuminant: evidence from a computational study
College of Computer Science, Sichuan University（四川大学计算机学院）；School of Life Science and Technology, University of Electronic Science and Technology of China（电子科技大学生命科学与技术学院）
This paper presents a novel design methodology for architecting a light-weight and faster DNN architecture for vision applications. The effectiveness of the architecture is demonstrated on Color-Constancy use case an inherent block in camera and imaging pipelines. Specifically, we present a multi-branch architecture that disassembles the contextual features and color properties from an image, and later combines them to predict a global property (e.g. Global Illumination). We also propose an implicit regularization technique by designing cross-branch regularization block that enables the networ…
Frankfurt Institute for Advanced Studies（法兰克福高等研究院）；Xidian-FIAS International Joint Research Center（西电-法兰克福高等研究院国际联合研究中心）；Justus Liebig University Giessen（吉森大学）
In this paper the use of Random Sprays Retinex (RSR) algorithm for global illumination estimation is proposed and its feasibility tested. Like other algorithms based on the Retinex model, RSR also provides local illumination estimation and brightness adjustment for each pixel and it is faster than other path-wise Retinex algorithms. As the assumption of the uniform illumination holds in many cases, it should be possible to use the mean of local illumination estimations of RSR as a global illumination estimation for images with (assumed) uniform illumination allowing also the accuracy to be eas…
INESC TEC；Tecnalia（特克纳利亚）；Computer Vision Center, Universitat Autónoma de Barcelona（巴塞罗那自治大学计算机视觉中心）；Faculdade de Engenharia, Universidade do Porto（波尔图大学工程学院）
Dermoscopic skin images are often obtained with different imaging devices, under varying acquisition conditions. In this work, instead of attempting to perform intensity and color normalization, we propose to leverage computational color constancy techniques to build an artificial data augmentation technique suitable for this kind of images. Specifically, we apply the \emph{shades of gray} color constancy technique to color-normalize the entire training set of images, while retaining the estimated illuminants. We then draw one sample from the distribution of training set illuminants and apply …
A statistical learning/inference framework for color demosaicing is presented. We start with simplistic assumptions about color constancy, and recast color demosaicing as a blind linear inverse problem: color parameterizes the unknown kernel, while brightness takes on the role of a latent variable. An expectation-maximization algorithm naturally suggests itself for the estimation of them both. Then, as we gradually broaden the family of hypothesis where color is learned, we let our demosaicing behave adaptively, in a manner that reflects our prior knowledge about the statistics of color images…
Institute for Information Transmission Problems of the Russian Academy of Sciences (IITP RAS)（俄罗斯科学院信息传输问题研究所）
Simple convolutional neural network was able to win ISISPA color constancy competition. Partial reimplementation of (Bianco, 2017) neural architecture would have shown even better results in this setup.
University of Amsterdam（阿姆斯特丹大学）；DUniversum（3DUniversum）
In this paper, we formulate the color constancy task as an image-to-image translation problem using GANs. By conducting a large set of experiments on different datasets, an experimental survey is provided on the use of different types of GANs to solve for color constancy i.e. CC-GANs (Color Constancy GANs). Based on the experimental review, recommendations are given for the design of CC-GAN architectures based on different criteria, circumstances and datasets.
Institute for Information Transmission Problems, RAS（俄罗斯科学院信息传输问题研究所）；University of Zagreb（萨格勒布大学）
Computational color constancy has the important task of reducing the influence of the scene illumination on the object colors. As such, it is an essential part of the image processing pipelines of most digital cameras. One of the important parts of the computational color constancy is illumination estimation, i.e. estimating the illumination color. When an illumination estimation method is proposed, its accuracy is usually reported by providing the values of error metrics obtained on the images of publicly available datasets. However, over time it has been shown that many of these datasets hav…
The pipelines of digital cameras contain a part for computational color constancy, which aims to remove the influence of the illumination on the scene colors. One of the best known and most widely used benchmark datasets for this problem is the Color Checker dataset. However, due to the improper handling of the black level in its images, this dataset has been widely misused and while some recent publications tried to alleviate the problem, they nevertheless erred and created additional wrong data. This paper gives a history of the Color Checker dataset usage, it describes the origins and reaso…
Universitat Pompeu Fabra（庞培法布拉大学）；University of East Anglia（东英吉利大学）
Visual illusions teach us that what we see is not always what it is represented in the physical world. Its special nature make them a fascinating tool to test and validate any new vision model proposed. In general, current vision models are based on the concatenation of linear convolutions and non-linear operations. In this paper we get inspiration from the similarity of this structure with the operations present in Convolutional Neural Networks (CNNs). This motivated us to study if CNNs trained for low-level visual tasks are deceived by visual illusions. In particular, we show that CNNs train…
