NightVision: Generating Nighttime Satellite Imagery from Infra-Red Observations
Fraunhofer Center Machine Learning（弗劳恩霍夫机器学习中心）；University of Kaiserlautern（凯泽斯劳滕大学）；University of Oxford（牛津大学）；CEA Saclay（萨克雷法国原子能和替代能源委员会）；Sorbonne Université（索邦大学）；University of Edinburgh（爱丁堡大学）；Dresden University of Technology（德累斯顿工业大学）；University of Bern（伯尔尼大学）；North Carolina State University（北卡罗来纳州立大学）
Paula Harder, William Jones, Redouane Lguensat, Shahine Bouabid, James Fulton, Dánell Quesada-Chacón, Aris Marcolongo, Sofija Stefanović, Yuhan Rao, Peter Mansh… 展开作者
Paula Harder, William Jones, Redouane Lguensat, Shahine Bouabid, James Fulton, Dánell Quesada-Chacón, Aris Marcolongo, Sofija Stefanović, Yuhan Rao, Peter Manshausen, Duncan Watson-Parris
The recent explosion in applications of machine learning to satellite imagery often rely on visible images and therefore suffer from a lack of data during the night. The gap can be filled by employing available infra-red observations to generate visible images. This work presents how deep learning can be applied successfully to create those images by using U-Net based architectures. The proposed methods show promising results, achieving a structural similarity index (SSIM) up to 86\% on an independent test set and providing visually convincing output images, generated from infra-red observations.
Beijing Institute of Technology（北京理工大学）；Yantai IRay Technologies Lt. Co.（烟台艾睿光电科技有限公司）
Infrared cameras are often utilized to enhance the night vision since the visible light cameras exhibit inferior efficacy without sufficient illumination. However, infrared data possesses inadequate color contrast and representation ability attributed to its intrinsic heat-related imaging principle. This makes it arduous to capture and analyze information for human beings, meanwhile hindering its application. Although, the domain gaps between unpaired nighttime infrared and daytime visible videos are even huger than paired ones that captured at the same time, establishing an effective translat…
University of Electronic Science and Technology of China（电子科技大学）；Beijing Institute of Basic Medical Sciences（北京基础医学研究所）
Benefitting from insensitivity to light and high penetration of foggy environments, infrared cameras are widely used for sensing in nighttime traffic scenes. However, the low contrast and lack of chromaticity of thermal infrared (TIR) images hinder the human interpretation and portability of high-level computer vision algorithms. Colorization to translate a nighttime TIR image into a daytime color (NTIR2DC) image may be a promising way to facilitate nighttime scene perception. Despite recent impressive advances in image translation, semantic encoding entanglement and geometric distortion in th…
University of Electronic Science and Technology of China（电子科技大学）
Nighttime thermal infrared (NTIR) image colorization, also known as translation of NTIR images into daytime color images (NTIR2DC), is a promising research direction to facilitate nighttime scene perception for humans and intelligent systems under unfavorable conditions (e.g., complete darkness). However, previously developed methods have poor colorization performance for small sample classes. Moreover, reducing the high confidence noise in pseudo-labels and addressing the problem of image gradient disappearance during translation are still under-explored, and keeping edges from being distorte…
University of South Carolina（南卡罗来纳大学）；East China Normal University（华东师范大学）
Nighttime satellite imagery has been applied in a wide range of fields. However, our limited understanding of how observed light intensity is formed and whether it can be simulated greatly hinders its further application. This study explores the potential of conditional Generative Adversarial Networks (cGAN) in translating multispectral imagery to nighttime imagery. A popular cGAN framework, pix2pix, was adopted and modified to facilitate this translation using gridded training image pairs derived from Landsat 8 and Visible Infrared Imaging Radiometer Suite (VIIRS). The results of this study p…
INAF-IASF Palermo, Istituto di Astrofisica e Fisica Cosmica（意大利国家天体物理研究所-巴勒莫宇宙天体物理研究所）；Università di Torino（都灵大学）
EUSO-SPB1 was released on April 24th, 2017, from the NASA balloon launch site in Wanaka (New Zealand) and landed on the South Pacific Ocean on May 7th. The data collected by the instruments onboard the balloon were analyzed to search UV pulse signatures of UHECR (Ultra High Energy Cosmic Rays) air showers. Indirect measurements of UHECRs can be affected by cloud presence during nighttime, therefore it is crucial to know the meteorological conditions during the observation period of the detector. During the flight, the onboard EUSO-SPB1 UCIRC camera (University of Chicago Infra-Red Camera), acq…
Beihang University（北京航空航天大学）；National Institute of Informatics（国立信息学研究所）；Peng Cheng Laboratory（鹏城实验室）
The current industry practice for 24-hour outdoor imaging is to use a silicon camera supplemented with near-infrared (NIR) illumination. This will result in color images with poor contrast at daytime and absence of chrominance at nighttime. For this dilemma, all existing solutions try to capture RGB and NIR images separately. However, they need additional hardware support and suffer from various drawbacks, including short service life, high price, specific usage scenario, etc. In this paper, we propose a novel and integrated enhancement solution that produces clear color images, whether at abu…
Puerto Rico suffered severe damage from the category 5 hurricane (Maria) in September 2017. Total monetary damages are estimated to be ~92 billion USD, the third most costly tropical cyclone in US history. The response to this damage has been tempered and slow moving, with recent estimates placing 45% of the population without power three months after the storm. Consequently, we developed a unique data-fusion mapping approach called the Urban Development Index (UDI) and new open source tool, Comet Time Series (CometTS), to analyze the recovery of electricity and infrastructure in Puerto Rico. …
The UN Sustainable Development Goals allude to the importance of infrastructure quality in three of its seventeen goals. However, monitoring infrastructure quality in developing regions remains prohibitively expensive and impedes efforts to measure progress toward these goals. To this end, we investigate the use of widely available remote sensing data for the prediction of infrastructure quality in Africa. We train a convolutional neural network to predict ground truth labels from the Afrobarometer Round 6 survey using Landsat 8 and Sentinel 1 satellite imagery. Our best models predict infrast…
Surveillance based on Computer Vision has become a major necessity in current era. Most of the surveillance systems operate on visible light imaging, but performance based on visible light imaging is limited due to some factors like variation in light intensity during the daytime. The matter of concern lies in the need for processing images in low light, such as in the need of nighttime surveillance. In this paper, we have proposed a novel approach for human detection using FLIR(Forward Looking Infrared) camera. As the principle involves sensing based on thermal radiation in the Near IR Region…
Modern-day surveillance systems perform person recognition using deep learning-based face verification networks. Most state-of-the-art facial verification systems are trained using visible spectrum images. But, acquiring images in the visible spectrum is impractical in scenarios of low-light and nighttime conditions, and often images are captured in an alternate domain such as the thermal infrared domain. Facial verification in thermal images is often performed after retrieving the corresponding visible domain images. This is a well-established problem often known as the Thermal-to-Visible (T2…
