The exposed cameras of UAV can shake, shift, or even malfunction under the influence of harsh weather, while the add-on devices (Dupont lines) are very vulnerable to damage.
We can place a low-cost T-OLED overlay around the camera to protect it, but this would also introduce image degradation issues.
In particular, the temperature variations in the atmosphere can create mist that adsorbs to the T-OLED, which can cause secondary disasters (i.e., more severe image degradation) during the UAV's filming process.
To solve the image degradation problem caused by overlaying T-OLEDs, in this paper we propose a new method to enhance the visual experience by enhancing the texture and color of images.
Specifically, our method trains a lightweight network to estimate a low-rank affine grid on the input image, and then utilizes the grid to enhance the input image at block granularity.
The advantages of our method are that no reference image is required and the loss function is developed from visual experience.
In addition, our model can perform high-quality recovery of images of arbitrary resolution in real time.
In the end, the limitations of our model and the collected datasets (including the daytime and nighttime scenes) are discussed.
Compositional-Degradation UAV Image Restoration: Conditional Decoupled MoE Network and A Benchmark
Key Laboratory of Intelligent Computing & Signal Processing (Anhui University), Ministry of Education（教育部智能计算与信号处理重点实验室（安徽大学））；Anhui Provincial Key Laboratory of Multimodal Cognitive Computation（安徽省多模态认知计算重点实验室）；School of Artificial Intelligence, Anhui University（安徽大学人工智能学院）；School of Computer Science and Technology, Anhui University（安徽大学计算机科学与技术学院）
Under-display camera (UDC) is a novel technology that can make digital imaging experience in handheld devices seamless by providing large screen-to-body ratio. UDC images are severely degraded owing to their positioning under a display screen. This work addresses the restoration of images degraded as a result of UDC imaging. Two different networks are proposed for the restoration of images taken with two types of UDC technologies. The first method uses a pyramidal dilated convolution within a wavelet decomposed convolutional neural network for pentile-organic LED (P-OLED) based display system.…
The new trend of full-screen devices encourages us to position a camera behind a screen. Removing the bezel and centralizing the camera under the screen brings larger display-to-body ratio and enhances eye contact in video chat, but also causes image degradation. In this paper, we focus on a newly-defined Under-Display Camera (UDC), as a novel real-world single image restoration problem. First, we take a 4k Transparent OLED (T-OLED) and a phone Pentile OLED (P-OLED) and analyze their optical systems to understand the degradation. Second, we design a Monitor-Camera Imaging System (MCIS) for eas…
A Task-Driven Evaluation of UAV Detection and Tracking under Synthetic Fog
Center for Aerospace Research (CfAR), University of Victoria（维多利亚大学航空航天研究中心）；Advanced Control and Intelligent Systems (ACIS) Lab, University of Victoria（维多利亚大学先进控制与智能系统实验室）
Indian Institute of Technology Madras（印度理工学院马德拉斯分校）
Under Display Cameras present a promising opportunity for phone manufacturers to achieve bezel-free displays by positioning the camera behind semi-transparent OLED screens. Unfortunately, such imaging systems suffer from severe image degradation due to light attenuation and diffraction effects. In this work, we present Deep Atrous Guided Filter (DAGF), a two-stage, end-to-end approach for image restoration in UDC systems. A Low-Resolution Network first restores image quality at low-resolution, which is subsequently used by the Guided Filter Network as a filtering input to produce a high-resolu…
This paper is the report of the first Under-Display Camera (UDC) image restoration challenge in conjunction with the RLQ workshop at ECCV 2020. The challenge is based on a newly-collected database of Under-Display Camera. The challenge tracks correspond to two types of display: a 4k Transparent OLED (T-OLED) and a phone Pentile OLED (P-OLED). Along with about 150 teams registered the challenge, eight and nine teams submitted the results during the testing phase for each track. The results in the paper are state-of-the-art restoration performance of Under-Display Camera Restoration. Datasets an…
Bridge Damage Detection from Low-Light UAV Imagery via Degradation-Aware Mixture-of-Experts Enhancement
School of Computer Science and Engineering, University of Electronic Science and Technology of China（电子科技大学计算机科学与工程学院）；Sustainability X-Lab, The University of Hong Kong（香港大学可持续发展X实验室）；School of Architecture, Building, and Civil Engineering, Loughborough University（拉夫堡大学建筑、建造与土木工程学院）；Department of Engineering Science, University of Oxford（牛津大学工程科学系）；Department of Civil and Environmental Engineering, The Hong Kong University of Science and Technology（香港科技大学土木与环境工程系）
CoRE-UIR: Prior-guided common and residual experts for efficient all-in-one remote sensing image restoration
School of Geodesy and Geomatics, Wuhan University（武汉大学测绘学院）；School of Artificial Intelligence, Wuhan University（武汉大学人工智能学院）；State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University（武汉大学测绘遥感信息工程国家重点实验室）
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；University of Chinese Academy of Sciences（中国科学院大学）；University of Macau（澳门大学）；Shanghai AI Laboratory（上海人工智能实验室）
Under-Display Camera (UDC) has been widely exploited to help smartphones realize full screen display. However, as the screen could inevitably affect the light propagation process, the images captured by the UDC system usually contain flare, haze, blur, and noise. Particularly, flare and blur in UDC images could severely deteriorate the user experience in high dynamic range (HDR) scenes. In this paper, we propose a new deep model, namely UDC-UNet, to address the UDC image restoration problem with the known Point Spread Function (PSF) in HDR scenes. On the premise that Point Spread Function (PSF…
Nanyang Technological University（南洋理工大学）；SenseBrain；Shanghai AI Laboratory（上海人工智能实验室）；SenseTime Research（商汤科技研究院）
Developing and integrating advanced image sensors with novel algorithms in camera systems are prevalent with the increasing demand for computational photography and imaging on mobile platforms. However, the lack of high-quality data for research and the rare opportunity for in-depth exchange of views from industry and academia constrain the development of mobile intelligent photography and imaging (MIPI). To bridge the gap, we introduce the first MIPI challenge including five tracks focusing on novel image sensors and imaging algorithms. In this paper, we summarize and review the Under-Display…
Segmentation Guided Sparse Transformer for Under-Display Camera Image Restoration
Shenzhen Campus of Sun Yat-sen University（中山大学深圳校区）；Hong Kong University of Science and Technology（香港科技大学）；State Key Laboratory for Novel Software Technology, Nanjing University（南京大学计算机软件新技术国家重点实验室）；Samsung Research China - Beijing (SRC-B)（三星中国研究院北京分部）；Harbin Institute of Technology, Shenzhen（哈尔滨工业大学深圳校区）；SAMSUNG Electronics（三星电子）
University of Notre Dame（圣母大学）；University of Ljubljana（卢布尔雅那大学）
Advances in image restoration and enhancement techniques have led to discussion about how such algorithmscan be applied as a pre-processing step to improve automatic visual recognition. In principle, techniques like deblurring and super-resolution should yield improvements by de-emphasizing noise and increasing signal in an input image. But the historically divergent goals of the computational photography and visual recognition communities have created a significant need for more work in this direction. To facilitate new research, we introduce a new benchmark dataset called UG^2, which contain…
Under-display cameras have been proposed in recent years as a way to reduce the form factor of mobile devices while maximizing the screen area. Unfortunately, placing the camera behind the screen results in significant image distortions, including loss of contrast, blur, noise, color shift, scattering artifacts, and reduced light sensitivity. In this paper, we propose an image-restoration pipeline that is ISP-agnostic, i.e. it can be combined with any legacy ISP to produce a final image that matches the appearance of regular cameras using the same ISP. This is achieved with a deep learning app…
University of Cambridge（剑桥大学）；Royal Society for Protection of Birds（英国皇家鸟类保护协会）；Jambi University（占碑大学）；PT Restorasi Ekosistem Indonesia（印度尼西亚生态系统恢复公司）
Logged forests cover four million square kilometres of the tropics and restoring these forests is essential if we are to avoid the worst impacts of climate change, yet monitoring recovery is challenging. Tracking the abundance of visually identifiable, early-successional species enables successional status and thereby restoration progress to be evaluated. Here we present a new pipeline, SLIC-UAV, for processing Unmanned Aerial Vehicle (UAV) imagery to map early-successional species in tropical forests. The pipeline is novel because it comprises: (a) a time-efficient approach for labelling crow…
Power Line Aerial Image Restoration under dverse Weather: Datasets and Baselines
School of Electrical Engineering and Automation, Nantong University（南通大学电气工程及其自动化学院）；School of Artificial Intelligence and Computer Science, Nantong University（南通大学人工智能与计算机科学学院）；School of Information Science and Technology, Nantong University（南通大学信息科学与技术学院）；College of Computer and Software Engineering, Hohai University（河海大学计算机与软件学院）；School of Electronic and Information Engineering, Suzhou University of Science and Technology（苏州科技大学电子与信息工程学院）；School of Information and Communication Technology, Griffith University（格里菲斯大学信息与通信技术学院）
CAWM-Mamba: A unified model for infrared-visible image fusion and compound adverse weather restoration
School of Physics（物理学院）；Optoelectronic Engineering, Foshan University, Foshan 528225, China（光学电子工程学院，佛山大学，佛山528225，中国）；Guangdong-HongKong-Macao Joint Laboratory for Intelligent Micro-Nano Optoelectronic Technology, Foshan 528225, China（粤港澳联合智能微纳光电子技术实验室，佛山528225，中国）；School of Mechanical Electronic（机械电子学院）；Information Engineering, China University of Mining and Technology, Beijing 100083, China（信息工程学院，中国矿业大学，北京100083，中国）
School of Robotics, Hunan University（湖南大学机器人学院）；College of Electrical and Information Engineering, Hunan University（湖南大学电气与信息工程学院）；Technical University Munich（慕尼黑工业大学）
State Key Lab for Novel Software Technology, Nanjing University（南京大学现代软件技术国家重点实验室）；Australian National University（澳大利亚国立大学）；Shenzhen Campus of Sun Yat-sen University（中山大学深圳校区）；Samsung Research China-Beijing (SRC-B)（三星中国研究院北京分部）；Imperial College London（伦敦帝国学院）
School of Geodesy and Geomatics, Wuhan University（武汉大学测绘学院）；Leibniz University Hannover（汉诺威莱布尼茨大学）
Many visual simultaneous localization and mapping (SLAM) systems have been shown to be accurate and robust, and have real-time performance capabilities on both indoor and ground datasets. However, these methods can be problematic when dealing with aerial frames captured by a camera mounted on an unmanned aerial vehicle (UAV) because the flight height of the UAV can be difficult to control and is easily affected by the environment.To cope with the case of lost tracking, many visual SLAM systems employ a relocalization strategy. This involves the tracking thread continuing the online working by …
Recent development of Under-Display Camera (UDC) systems provides a true bezel-less and notch-free viewing experience on smartphones (and TV, laptops, tablets), while allowing images to be captured from the selfie camera embedded underneath. In a typical UDC system, the microstructure of the semi-transparent organic light-emitting diode (OLED) pixel array attenuates and diffracts the incident light on the camera, resulting in significant image quality degradation. Oftentimes, noise, flare, haze, and blur can be observed in UDC images. In this work, we aim to analyze and tackle the aforemention…
Peking University（北京大学）；Microsoft Research Asia（微软亚洲研究院）；Peng Cheng Laboratory（鹏城实验室）
Image rescaling is a commonly used bidirectional operation, which first downscales high-resolution images to fit various display screens or to be storage- and bandwidth-friendly, and afterward upscales the corresponding low-resolution images to recover the original resolution or the details in the zoom-in images. However, the non-injective downscaling mapping discards high-frequency contents, leading to the ill-posed problem for the inverse restoration task. This can be abstracted as a general image degradation-restoration problem with information loss. In this work, we propose a novel inverti…
National University of Singapore（新加坡国立大学）；Wuhan Cobot Technology Co., Ltd.（武汉科博特科技有限公司）；Huazhong University of Science and Technology（华中科技大学）
Unwanted nonlinear gamma distortion frequently occurs in a great diversity of images during the procedures of image acquisition, processing, and/or display. And the gamma distortion often varies with capture setup change and luminance variation. Blind inverse gamma correction, which automatically determines a proper restoration gamma value from a given image, is of paramount importance to attenuate the distortion. For blind inverse gamma correction, an adaptive gamma transformation method (AGT-ME) is proposed directly from a maximized differential entropy model. And the corresponding optimizat…
Under-display camera (UDC) provides an elegant solution for full-screen smartphones. However, UDC captured images suffer from severe degradation since sensors lie under the display. Although this issue can be tackled by image restoration networks, these networks require large-scale image pairs for training. To this end, we propose a modular network dubbed MPGNet trained using the generative adversarial network (GAN) framework for simulating UDC imaging. Specifically, we note that the UDC imaging degradation process contains brightness attenuation, blurring, and noise corruption. Thus we model …
University of Notre Dame（圣母大学）；University of Texas at Austin（德克萨斯大学奥斯汀分校）
How can we effectively engineer a computer vision system that is able to interpret videos from unconstrained mobility platforms like UAVs? One promising option is to make use of image restoration and enhancement algorithms from the area of computational photography to improve the quality of the underlying frames in a way that also improves automatic visual recognition. Along these lines, exploratory work is needed to find out which image pre-processing algorithms, in combination with the strongest features and supervised machine learning approaches, are good candidates for difficult scenarios …
Combined Flicker-banding and Moire Removal for Screen-Captured Images
Shanghai Jiao Tong University, Shanghai, China（上海交通大学）；Central Media Technology Institute, Huawei, China（华为中央媒体技术研究所）；Department of XXX, University of YYY, Location, Country（YYY大学XXX系）；School of ZZZ, Institute of WWW, Location, Country（ZZZ学院WWW研究所）
From Fog Chamber to Aircraft Window: Pixel-Registered Imaging and Synthetic Fine-Tuning Enable Cross-Domain Defogging
Department of Electrical and Computer Engineering, University of Utah（犹他大学电气与计算机工程系）；U.S. Naval Research Laboratory Remote Sensing Division（美国海军研究实验室远程传感部门）
UDC-VIT: A Real-World Video Dataset for Under-Display Cameras
Dept. of Data Science, Seoul National University（数据科学系，首尔国立大学）；Dept. of Computer Science and Engineering, Seoul National University（计算机科学与工程系，首尔国立大学）；Research Center, Samsung Display Co., Ltd.（三星显示公司研究部）；Mobile Display Electronics Development Team, Samsung Display Co., Ltd.（三星显示公司移动显示电子开发团队）
Instituto de Matemática e Estatística, Universidade de São Paulo（圣保罗大学数学与统计研究所）；Universidade Federal de São Carlos（圣卡洛斯联邦大学）；Mayo Clinic College of Medicine（梅奥诊所医学院）；Universidade Federal de Alagoas（阿拉戈斯联邦大学）
Vibro-acoustography (VA) is a medical imaging method based on the difference-frequency generation produced by the mixture of two focused ultrasound beams. VA has been applied to different problems in medical imaging such as imaging bones, microcalcifications in the breast, mass lesions, and calcified arteries. The obtained images may have a resolution of 0.7--0.8 mm. Current VA systems based on confocal or linear array transducers generate C-scan images at the beam focal plane. Images on the axial plane are also possible, however the system resolution along depth worsens when compared to the l…
