Inverse Tone Mapping (ITM) methods attempt to reconstruct High Dynamic Range (HDR) information from Low Dynamic Range (LDR) image content. The dynamic range of well-exposed areas must be expanded and any missing information due to over/under-exposure must be recovered (hallucinated). The majority of methods focus on the former and are relatively successful, while most attempts on the latter are not of sufficient quality, even ones based on Convolutional Neural Networks (CNNs). A major factor for the reduced inpainting quality in some works is the choice of loss function. Work based on Generative Adversarial Networks (GANs) shows promising results for image synthesis and LDR inpainting, suggesting that GAN losses can improve inverse tone mapping results. This work presents a GAN-based method that hallucinates missing information from badly exposed areas in LDR images and compares its efficacy with alternative variations. The proposed method is quantitatively competitive with state-of-the-art inverse tone mapping methods, providing good dynamic range expansion for well-exposed areas and plausible hallucinations for saturated and under-exposed areas. A density-based normalisation method, targeted for HDR content, is also proposed, as well as an HDR data augmentation method targeted for HDR hallucination.
Trinity College Dublin（都柏林圣三一学院）；Laboratoire des Signaux et Systémes（信号与系统实验室）；CNRS（法国国家科学研究中心）；CentraleSupelec（巴黎中央理工-高等电力学院）；Université Paris-Sud（巴黎南大学）；Ecole des Ponts ParisTech（巴黎桥路学院）；Université Paris-Est（巴黎东大学）
A computationally fast tone mapping operator (TMO) that can quickly adapt to a wide spectrum of high dynamic range (HDR) content is quintessential for visualization on varied low dynamic range (LDR) output devices such as movie screens or standard displays. Existing TMOs can successfully tone-map only a limited number of HDR content and require an extensive parameter tuning to yield the best subjective-quality tone-mapped output. In this paper, we address this problem by proposing a fast, parameter-free and scene-adaptable deep tone mapping operator (DeepTMO) that yields a high-resolution and …
CYENS Centre of Excellence（CYENS卓越中心）；Institute of Information Science and Technologies of the National Research Council of Italy (ISTI-CNR)（意大利国家研究委员会信息科学与技术研究所）
The prime goal of digital imaging techniques is to reproduce the realistic appearance of a scene. Low Dynamic Range (LDR) cameras are incapable of representing the wide dynamic range of the real-world scene. The captured images turn out to be either too dark (underexposed) or too bright (overexposed). Specifically, saturation in overexposed regions makes the task of reconstructing a High Dynamic Range (HDR) image from single LDR image challenging. In this paper, we propose a deep learning based approach to recover details in the saturated areas while reconstructing the HDR image. We formulate …
Max Planck Institute for Informatics（马克斯·普朗克信息学研究所）；Universidad de Zaragoza（萨拉戈萨大学）
Most in-the-wild images are stored in Low Dynamic Range (LDR) form, serving as a partial observation of the High Dynamic Range (HDR) visual world. Despite limited dynamic range, these LDR images are often captured with different exposures, implicitly containing information about the underlying HDR image distribution. Inspired by this intuition, in this work we present, to the best of our knowledge, the first method for learning a generative model of HDR images from in-the-wild LDR image collections in a fully unsupervised manner. The key idea is to train a generative adversarial network (GAN) …
Mapping a single exposure low dynamic range (LDR) image into a high dynamic range (HDR) is considered among the most strenuous image to image translation tasks due to exposure-related missing information. This study tackles the challenges of single-shot LDR to HDR mapping by proposing a novel two-stage deep network. Notably, our proposed method aims to reconstruct an HDR image without knowing hardware information, including camera response function (CRF) and exposure settings. Therefore, we aim to perform image enhancement task like denoising, exposure correction, etc., in the first stage. Add…
City University of Hong Kong（香港城市大学）；Dalian University of Technology（大连理工大学）；Tencent AI Lab（腾讯AI实验室）
Image correction aims to adjust an input image into a visually pleasing one. Existing approaches are proposed mainly from the perspective of image pixel manipulation. They are not effective to recover the details in the under/over exposed regions. In this paper, we revisit the image formation procedure and notice that the missing details in these regions exist in the corresponding high dynamic range (HDR) data. These details are well perceived by the human eyes but diminished in the low dynamic range (LDR) domain because of the tone mapping process. Therefore, we formulate the image correction…
Tokyo Metropolitan College of Industrial Technology（东京都立产业技术大学院大学）；Tokyo Metropolitan University（东京都立大学）
We proposed a lossless two-layer HDR coding method using a histogram packing technique. The proposed method was demonstrated to outperform the normative JPEG XT encoder, under the use of the default tone-mapping operator. However, the performance under various tone-mapping operators has not been discussed. In this paper, we aim to compare the performance of the proposed method with that of the JPEG XT encoder under the use of various tone-mapping operators to clearly show the characteristic difference between them.
An image processing unit (IPU), or image signal processor (ISP) for high dynamic range (HDR) imaging usually consists of demosaicing, white balancing, lens shading correction, color correction, denoising, and tone-mapping. Besides noise from the imaging sensors, almost every step in the ISP introduces or amplifies noise in different ways, and denoising operators are designed to reduce the noise from these sources. Designed for dynamic range compressing, tone-mapping operators in an ISP can significantly amplify the noise level, especially for images captured in low-light conditions, making den…
We describe a deep high-dynamic-range (HDR) image tone mapping operator that is computationally efficient and perceptually optimized. We first decompose an HDR image into a normalized Laplacian pyramid, and use two deep neural networks (DNNs) to estimate the Laplacian pyramid of the desired tone-mapped image from the normalized representation. We then end-to-end optimize the entire method over a database of HDR images by minimizing the normalized Laplacian pyramid distance (NLPD), a recently proposed perceptual metric. Qualitative and quantitative experiments demonstrate that our method produc…
University of Nottingham Ningbo China（宁波诺丁汉大学）；Shenzhen University（深圳大学）；Southwestern University of Finance and Economics（西南财经大学）；University of Nottingham（诺丁汉大学）
Building on crucial insights into the determining factors of the visual integrity of an image and the property of deep convolutional neural network (CNN), we have developed the Deep Feature Consistent Deep Image Transformation (DFC-DIT) framework which unifies challenging one-to-many mapping image processing problems such as image downscaling, decolorization (colour to grayscale conversion) and high dynamic range (HDR) image tone mapping. We train one CNN as a non-linear mapper to transform an input image to an output image following what we term the deep feature consistency principle which is…
The Chinese University of Hong Kong（香港中文大学）；Caritas Institute of Higher Education（明爱专上学院）
Although digital cameras can acquire high-dynamic range (HDR) images, the captured HDR information are mostly quantized to low-dynamic range (LDR) images for display compatibility and compact storage. In this paper, we propose an invertible tone mapping method that converts the multi-exposure HDR to a true LDR (8-bit per color channel) and reserves the capability to accurately restore the original HDR from this {\em invertible LDR}. Our invertible LDR can mimic the appearance of a user-selected tone mapping style. It can be shared over any existing social network platforms that may re-encode o…
Korea Advanced Institute of Science and Technology（韩国科学技术院）
Recent modern displays are now able to render high dynamic range (HDR), high resolution (HR) videos of up to 8K UHD (Ultra High Definition). Consequently, UHD HDR broadcasting and streaming have emerged as high quality premium services. However, due to the lack of original UHD HDR video content, appropriate conversion technologies are urgently needed to transform the legacy low resolution (LR) standard dynamic range (SDR) videos into UHD HDR versions. In this paper, we propose a joint super-resolution (SR) and inverse tone-mapping (ITM) framework, called Deep SR-ITM, which learns the direct ma…
Real-Time Scene-Adaptive Tone Mapping for High-Dynamic Range Object Detection
School of Data Science, The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳）数据科学学院）；Tianmushan Laboratory, Beihang University（北京航空航天大学天目山实验室）；School of Artificial Intelligence, Nanjing University of Information Science and Technology（南京信息工程大学人工智能学院）；Department of Automation, Tsinghua University（清华大学自动化系）；Point Spread Technology（点扩散科技）
Korea Advanced Institute of Science and Technology（韩国科学技术院）
Joint learning of super-resolution (SR) and inverse tone-mapping (ITM) has been explored recently, to convert legacy low resolution (LR) standard dynamic range (SDR) videos to high resolution (HR) high dynamic range (HDR) videos for the growing need of UHD HDR TV/broadcasting applications. However, previous CNN-based methods directly reconstruct the HR HDR frames from LR SDR frames, and are only trained with a simple L2 loss. In this paper, we take a divide-and-conquer approach in designing a novel GAN-based joint SR-ITM network, called JSI-GAN, which is composed of three task-specific subnets…
Pohang University of Science and Technology (POSTECH)（浦项科技大学（POSTECH））；Yonsei University（延世大学）
We propose high dynamic range (HDR) radiance fields, HDR-Plenoxels, that learn a plenoptic function of 3D HDR radiance fields, geometry information, and varying camera settings inherent in 2D low dynamic range (LDR) images. Our voxel-based volume rendering pipeline reconstructs HDR radiance fields with only multi-view LDR images taken from varying camera settings in an end-to-end manner and has a fast convergence speed. To deal with various cameras in real-world scenarios, we introduce a tone mapping module that models the digital in-camera imaging pipeline (ISP) and disentangles radiometric s…
National Taiwan University（国立台湾大学）；MediaTek Inc.（联发科技股份有限公司）
Tone-mapping plays an essential role in high dynamic range (HDR) imaging. It aims to preserve visual information of HDR images in a medium with a limited dynamic range. Although many works have been proposed to provide tone-mapped results from HDR images, most of them can only perform tone-mapping in a single pre-designed way. However, the subjectivity of tone-mapping quality varies from person to person, and the preference of tone-mapping style also differs from application to application. In this paper, a learning-based multimodal tone-mapping method is proposed, which not only achieves exce…
Lahore University of Management Sciences（拉合尔管理科学大学）；CECOS University（塞科斯大学）；Georgia State University（乔治亚州立大学）
High Dynamic Range (HDR) images are the ones that contain a greater range of luminosity as compared to the standard images. HDR images have a higher detail and clarity of structure, objects, and color, which the standard images lack. HDR images are useful in capturing scenes that pose high brightness, darker areas, and shadows, etc. An HDR image comprises multiple narrow-range-exposure images combined into one high-quality image. As these HDR images cannot be displayed on standard display devices, the real challenge comes while converting these HDR images to Low dynamic range (LDR) images. The…
Comenius University Bratislava（布拉迪斯拉发夸美纽斯大学）；Technicolor Research & Innovation（特艺集团研究与创新部）；University of Girona（赫罗纳大学）；Middle East Technical University（中东技术大学）；ISTI-CNR（意大利国家研究委员会信息科学与技术研究所）
Few tone mapping operators (TMOs) take color management into consideration, limiting compression to luminance values only. This may lead to changes in image chroma and hues which are typically managed with a post-processing step. However, current post-processing techniques for tone reproduction do not explicitly consider the target display gamut. Gamut mapping on the other hand, deals with mapping images from one color gamut to another, usually smaller, gamut but has traditionally focused on smaller scale, chromatic changes. In this context, we present a novel gamut and tone management framewo…
High Dynamic Range Image Reconstruction via Deep Explicit Polynomial Curve Estimation
Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Hong Kong University of Science and Technology（香港科技大学）；SmartMore Technology（思谋科技）；Northwestern Polytechnical University（西北工业大学）；Zhejiang Lab（之江实验室）；Zhejiang University（浙江大学）
Max-Planck-Institut für Informatik（马克斯·普朗克计算机科学研究所）；Universidad de Zaragoza（萨拉戈萨大学）
High Dynamic Range (HDR) content is becoming ubiquitous due to the rapid development of capture technologies. Nevertheless, the dynamic range of common display devices is still limited, therefore tone mapping (TM) remains a key challenge for image visualization. Recent work has demonstrated that neural networks can achieve remarkable performance in this task when compared to traditional methods, however, the quality of the results of these learning-based methods is limited by the training data. Most existing works use as training set a curated selection of best-performing results from existing…
University of Science and Technology of China（中国科学技术大学）；Huawei Noah’s Ark Lab（华为诺亚方舟实验室）
Existing deep learning based HDRTV reconstruction methods assume one kind of tone mapping operators (TMOs) as the degradation procedure to synthesize SDRTV-HDRTV pairs for supervised training. In this paper, we argue that, although traditional TMOs exploit efficient dynamic range compression priors, they have several drawbacks on modeling the realistic degradation: information over-preservation, color bias and possible artifacts, making the trained reconstruction networks hard to generalize well to real-world cases. To solve this problem, we propose a learning-based data synthesis approach to …
In this paper, we propose a deep snapshot high dynamic range (HDR) imaging framework that can effectively reconstruct an HDR image from the RAW data captured using a multi-exposure color filter array (ME-CFA), which consists of a mosaic pattern of RGB filters with different exposure levels. To effectively learn the HDR image reconstruction network, we introduce the idea of luminance normalization that simultaneously enables effective loss computation and input data normalization by considering relative local contrasts in the "normalized-by-luminance" HDR domain. This idea makes it possible to …
University of Warwick（华威大学）；University of the West of England（西英格兰大学）
High dynamic range (HDR) imaging provides the capability of handling real world lighting as opposed to the traditional low dynamic range (LDR) which struggles to accurately represent images with higher dynamic range. However, most imaging content is still available only in LDR. This paper presents a method for generating HDR content from LDR content based on deep Convolutional Neural Networks (CNNs) termed ExpandNet. ExpandNet accepts LDR images as input and generates images with an expanded range in an end-to-end fashion. The model attempts to reconstruct missing information that was lost fro…
The Chinese University of Hong Kong（香港中文大学）；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）
Tone mapping is a commonly used technique that maps the set of colors in high-dynamic-range (HDR) images to another set of colors in low-dynamic-range (LDR) images, to fit the need for print-outs, LCD monitors and projectors. Unfortunately, during the compression of dynamic range, the overall contrast and local details generally cannot be preserved simultaneously. Recently, with the increased use of stereoscopic devices, the notion of binocular tone mapping has been proposed in the existing research study. However, the existing research lacks the binocular perception study and is unable to gen…
High dynamic range (HDR) photography is becoming increasingly popular and available by DSLR and mobile-phone cameras. While deep neural networks (DNN) have greatly impacted other domains of image manipulation, their use for HDR tone-mapping is limited due to the lack of a definite notion of ground-truth solution, which is needed for producing training data. In this paper we describe a new tone-mapping approach guided by the distinct goal of producing low dynamic range (LDR) renditions that best reproduce the visual characteristics of native LDR images. This goal enables the use of an unpaired …
Learning Differential Pyramid Representation for Tone Mapping
School of Electrical and Information Engineering, Tianjin University, China（天津大学电气与信息工程学院）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；vivo Mobile Communication Co., Ltd（vivo移动通信有限公司）
ISTI-CNR（意大利国家研究委员会信息科学与技术研究所）；Warwick University（华威大学）；University of West England（西英格兰大学）
Recently, Deep Learning-based methods for inverse tone-mapping standard dynamic range (SDR) images to obtain high dynamic range (HDR) images have become very popular. These methods manage to fill over-exposed areas convincingly both in terms of details and dynamic range. Typically, these methods, to be effective, need to learn from large datasets and to transfer this knowledge to the network weights. In this work, we tackle this problem from a completely different perspective. What can we learn from a single SDR video? With the presented zero-shot approach, we show that, in many cases, a singl…
Perceptual Tone Mapping Model for High Dynamic Range Imaging
State Key Laboratory of Modern Optical Instrumentation, Zhejiang University（浙江大学现代光学仪器国家重点实验室）；Leeds Institute of Textiles and Colour, University of Leeds（利兹大学利兹纺织与色彩学院）
Pengcheng Laboratory（鹏城实验室）；University of Nottingham（诺丁汉大学）；Shenzhen University（深圳大学）
Single-image HDR reconstruction or inverse tone mapping (iTM) is a challenging task. In particular, recovering information in over-exposed regions is extremely difficult because details in such regions are almost completely lost. In this paper, we present a deep learning based iTM method that takes advantage of the feature extraction and mapping power of deep convolutional neural networks (CNNs) and uses a lightness prior to modulate the CNN to better exploit observations in the surrounding areas of the over-exposed regions to enhance the quality of HDR image reconstruction. Specifically, we i…
Indian Institute of Technology Gandhinagar（印度理工学院甘地讷格尔分校）
We propose novel methods of solving two tasks using Convolutional Neural Networks, firstly the task of generating HDR map of a static scene using differently exposed LDR images of the scene captured using conventional cameras and secondly the task of finding an optimal tone mapping operator that would give a better score on the TMQI metric compared to the existing methods. We quantitatively show the performance of our networks and illustrate the cases where our networks performs good as well as bad.
School of Information and Communication Engineering, University of Electronic Science and Technology of China（电子科技大学信息与通信工程学院）
High dynamic range (HDR) imaging has recently drawn much attention in multimedia community. In this paper, we proposed a HDR image forensics method based on convolutional neural network (CNN).To our best knowledge, this is the first time to apply deep learning method on HDR image forensics. The proposed algorithm uses CNN to distinguish HDR images generated by multiple low dynamic range (LDR) images from that expanded by single LDR image using inverse tone mapping (iTM). To do this, we learn the change of statistical characteristics extracted by the proposed CNN architectures and classify two …
Tone Matters: The Impact of Linguistic Tone on Hallucination in VLMs
Department of Computer Science and Technology, Kean University（计算机科学与技术系，凯恩大学）；Department of Computer Science and Engineering, University of Notre Dame（计算机科学与工程系，圣母大学）
University of Alberta（阿尔伯塔大学）；Dolby Laboratories Inc（杜比实验室公司）
High Dynamic Range (HDR) imaging is gaining increased attention due to its realistic content, for not only regular displays but also smartphones. Before sufficient HDR content is distributed, HDR visualization still relies mostly on converting Standard Dynamic Range (SDR) content. SDR images are often quantized, or bit depth reduced, before SDR-to-HDR conversion, e.g. for video transmission. Quantization can easily lead to banding artefacts. In some computing and/or memory I/O limited environment, the traditional solution using spatial neighborhood information is not feasible. Our method inclu…
