STEPS: Joint Self-supervised Nighttime Image Enhancement and Depth Estimation
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);Institute for AI Industry Research (AIR), Tsinghua University(清华大学人工智能产业研究院(AIR));Department of Computer Science and Technology, Tsinghua University(清华大学计算机科学与技术系);Xi’an Research Institute of High-Tech(西安高新技术研究所);Intel Labs China(英特尔中国研究院);Peking University(北京大学)
Self-supervised depth estimation draws a lot of attention recently as it can promote the 3D sensing capabilities of self-driving vehicles. However, it intrinsically relies upon the photometric consistency assumption, which hardly holds during nighttime. Although various supervised nighttime image enhancement methods have been proposed, their generalization performance in challenging driving scenarios is not satisfactory. To this end, we propose the first method that jointly learns a nighttime image enhancer and a depth estimator, without using ground truth for either task. Our method tightly entangles two self-supervised tasks using a newly proposed uncertain pixel masking strategy. This strategy originates from the observation that nighttime images not only suffer from underexposed regions but also from overexposed regions. By fitting a bridge-shaped curve to the illumination map distribution, both regions are suppressed and two tasks are bridged naturally. We benchmark the method on two established datasets: nuScenes and RobotCar and demonstrate state-of-the-art performance on both of them. Detailed ablations also reveal the mechanism of our proposal. Last but not least, to mitigate the problem of sparse ground truth of existing datasets, we provide a new photo-realistically enhanced nighttime dataset based upon CARLA. It brings meaningful new challenges to the community. Codes, data, and models are available at https://github.com/ucaszyp/STEPS.
Di$^2$CycleSB: Towards High-Quality Unsupervised Nighttime Visibility Enhancement via Schrödinger Bridge Transformer
Faculty of Data Science, City University of Macau(澳门城市大学数据科学学院);Department of Automation, Tsinghua University(清华大学自动化系);College of Computer Science and Software Engineering, Shenzhen University(深圳大学计算机与软件学院)
MambaNUT: Nighttime UAV Tracking via Mamba-based Adaptive Curriculum Learning
College of Computer Science and Engineering, Guilin University of Technology(桂林理工大学计算机科学与工程学院);School of Computer Science, Fudan University(复旦大学计算机科学学院);School of Artificial Intelligence, Sun Yat-sen University(中山大学人工智能学院)
Multi-Knowledge-oriented Nighttime Haze Imaging Enhancer for Vision-driven Intelligent Systems
School of Computer Science and Engineering(计算机科学与工程学院);University of Electronic Science and Technology of China(电子科技大学);Department of Logistics and Maritime Studies(物流与海事研究系);The Hong Kong Polytechnic University(香港理工大学);Chengdu Union Big Data Tech Inc.(成都联合大数据科技有限公司)
Event-RGB Adaptive Tracking for Nighttime Highway Perception
School of Intelligent Systems Engineering, Sun Yat-sen University(中山大学智能系统工程学院);School of Intelligence Science and Engineering, Harbin Institute of Technology(哈尔滨工业大学智能科学与工程学院)
Semi-LAR: Semi-supervised Contrastive Learning with Linear Attention for Removal of Nighttime Flares
School of Computer Science and Technology, Wuhan University of Science and Technology, Wuhan, China(武汉科技大学计算机科学与技术学院);Faculty of Computing, Harbin Institute of Technology, Harbin 150001, China(哈尔滨工业大学计算机学院);SRO department, Institute for Infocomm Research, A*STAR, Singapore(新加坡资讯通信研究院SRO部门)
Wangxuan Institute of Computer Technology, Peking University(北京大学王选计算机研究所);School of EECS, Peking University(北京大学电子工程与计算机科学学院);School of CS, Peking University(北京大学计算机科学学院);Peng Cheng Laboratory(鹏城实验室)
CLIP-Guided Data Augmentation for Night-Time Image Dehazing
Hangzhou Dianzi University(杭州电子科技大学);Jinan University(济南大学);Hefei University of Technology(合肥工业大学);Wuhan University(武汉大学);The University of Tokyo(东京大学)
Joint RGB-Spectral Decomposition Model Guided Image Enhancement in Mobile Photography
School of Electronic Science and Engineering, Nanjing University(南京大学电子科学与工程学院);Nanyang Technological University(南洋理工大学);Key Laboratory of Optoelectronic Devices and Systems with Extreme Performances of MOE, Nanjing University(南京大学极端性能光电器件与系统教育部重点实验室)
Joint Medical Image Enhancement and Segmentation with Diffusion-based Symbiotic Information Interaction
Shenzhen Research Institute, The Chinese University of Hong Kong(香港中文大学深圳研究院);University of Science and Technology of China(中国科学技术大学);Imperial Global Singapore, Imperial College London(伦敦帝国学院新加坡分校)
Joint Face Hallucination and Deblurring via Structure Generation and Detail Enhancement
Sensetime Research(商汤科技研究院);Tencent(腾讯);South China University of Technology(华南理工大学);University of Science and Technology of China(中国科学技术大学);University of California at Merced(加州大学默塞德分校);Tencent AI Lab(腾讯人工智能实验室);Nanjing University of Science and Technology(南京理工大学)
VJT: A Video Transformer on Joint Tasks of Deblurring, Low-light Enhancement and Denoising
The Chinese University of Hong Kong(香港中文大学);National University of Defense Technology(国防科技大学);City University of Hong Kong(香港城市大学);Nanjing University of Science and Technology(南京理工大学)