In image fusion tasks, images from different sources possess distinct characteristics. This has driven the development of numerous methods to explore better ways of fusing them while preserving their respective characteristics.Mamba, as a state space model, has emerged in the field of natural language processing. Recently, many studies have attempted to extend Mamba to vision tasks. However, due to the nature of images different from causal language sequences, the limited state capacity of Mamba weakens its ability to model image information. Additionally, the sequence modeling ability of Mamba is only capable of spatial information and cannot effectively capture the rich spectral information in images. Motivated by these challenges, we customize and improve the vision Mamba network designed for the image fusion task. Specifically, we propose the local-enhanced vision Mamba block, dubbed as LEVM. The LEVM block can improve local information perception of the network and simultaneously learn local and global spatial information. Furthermore, we propose the state sharing technique to enhance spatial details and integrate spatial and spectral information. Finally, the overall network is a multi-scale structure based on vision Mamba, called LE-Mamba. Extensive experiments show the proposed methods achieve state-of-the-art results on multispectral pansharpening and multispectral and hyperspectral image fusion datasets, and demonstrate the effectiveness of the proposed approach. Codes can be accessed at \url{https://github.com/294coder/Efficient-MIF}.
Low-light images are not conducive to human observation and computer vision algorithms due to their low visibility. Although many image enhancement techniques have been proposed to solve this problem, existing methods inevitably introduce contrast under- and over-enhancement. Inspired by human visual system, we design a multi-exposure fusion framework for low-light image enhancement. Based on the framework, we propose a dual-exposure fusion algorithm to provide an accurate contrast and lightness enhancement. Specifically, we first design the weight matrix for image fusion using illumination es…
This report presents the results of a multi-scale wavelet based scheme for single image de-hazing and underwater image enhancement. The scheme is fast and highly localized in addition to global enhancement of hazy images. A PDE-based formulation enables additional versatility as the iterative nature allows more flexibility for various types of images. Visual and objective results from experiments indicate that the proposed approach competes favourably or surpasses most of the state-of-the-art approaches.
FusionNet: Multi-model Linear Fusion Framework for Low-light Image Enhancement
Northwestern Polytechnical University（西北工业大学）；Shenzhen Research Institute of Northwestern Polytechnical University（西北工业大学深圳研究院）；Xi’an Institute of Optics and Precision Mechanics of CAS（中国科学院西安光学精密机械研究所）；Shanghai Institute of Satellite Engineering（上海航天工程研究所）
ASUS Intelligent Cloud Services, Asustek Computer Inc.（华硕电脑股份有限公司智能云服务）；National Taiwan University（国立台湾大学）
Underwater image enhancement is an important low-level computer vision task for autonomous underwater vehicles and remotely operated vehicles to explore and understand the underwater environments. Recently, deep convolutional neural networks (CNNs) have been successfully used in many computer vision problems, and so does underwater image enhancement. There are many deep-learning-based methods with impressive performance for underwater image enhancement, but their memory and model parameter costs are hindrances in practical application. To address this issue, we propose a lightweight adaptive f…
SSPM's College of Engineering, Kankavali（SSPM工程学院）
The idea of combining multiple image modalities to provide a single, enhanced image is well established different fusion methods have been proposed in literature. This paper is based on image fusion using laplacian pyramid and wavelet transform method. Images of same size are used for experimentation. Images used for the experimentation are standard images and averaging filter is used of equal weights in original images to burl. Performance of image fusion technique is measured by mean square error, normalized absolute error and peak signal to noise ratio. From the performance analysis it has …
AttentionLut: Attention Fusion-based Canonical Polyadic LUT for Real-time Image Enhancement
Institute of Image Communication and Network Engineering, Shanghai Jiao Tong University（上海交通大学图像通信与网络工程研究所）；Transsion（传音控股）；John Hopcroft Center, Shanghai Jiao Tong University（上海交通大学约翰·霍普克罗夫特中心）
This report presents the results of a proposed multi-scale fusion-based single image de-hazing algorithm, which can also be used for underwater image enhancement. Furthermore, the algorithm was designed for very fast operation and minimal run-time. The proposed scheme is the faster than existing algorithms for both de-hazing and underwater image enhancement and amenable to digital hardware implementation. Results indicate mostly consistent and good results for both categories of images when compared with other algorithms from the literature.
University of Science and Technology of China（中国科学技术大学）；Harbin Institute of Technology（哈尔滨工业大学）；Huawei Noah’s Ark Lab（华为诺亚方舟实验室）；University of Macau（澳门大学）；Shanghai AI Laboratory（上海人工智能实验室）；Huawei Cloud BU（华为云业务部）
Low-light video enhancement (LLVE) is an important yet challenging task with many applications such as photographing and autonomous driving. Unlike single image low-light enhancement, most LLVE methods utilize temporal information from adjacent frames to restore the color and remove the noise of the target frame. However, these algorithms, based on the framework of multi-frame alignment and enhancement, may produce multi-frame fusion artifacts when encountering extreme low light or fast motion. In this paper, inspired by the low latency and high dynamic range of events, we use synthetic events…
CLIP-Optimized Multimodal Image Enhancement via ISP-CNN Fusion for Coal Mine IoVT under Uneven Illumination
School of Artificial Intelligence, China University of Mining and Technology (Beijing)（中国矿业大学（北京）人工智能学院）；Ministry of Emergency Management Big Data Center（应急管理部大数据中心）
Various and different methods can be used to produce high-resolution multispectral images from high-resolution panchromatic image (PAN) and low-resolution multispectral images (MS), mostly on the pixel level. The Quality of image fusion is an essential determinant of the value of processing images fusion for many applications. Spatial and spectral qualities are the two important indexes that used to evaluate the quality of any fused image. However, the jury is still out of fused image's benefits if it compared with its original images. In addition, there is a lack of measures for assessing the…
Nanjing University of Information Science and Technology（南京信息工程大学）
Underwater image enhancement algorithms have attracted much attention in underwater vision task. However, these algorithms are mainly evaluated on different data sets and different metrics. In this paper, we set up an effective and pubic underwater test dataset named U45 including the color casts, low contrast and haze-like effects of underwater degradation and propose a fusion adversarial network for enhancing underwater images. Meanwhile, the well-designed the adversarial loss including Lgt loss and Lfe loss is presented to focus on image features of ground truth, and image features of the i…
Thresholded Cross-Attention for Reliable Intensity-Chromaticity Fusion in Low-Light Image Enhancement
School of Information Engineering, Guangdong University of Technology（广东工业大学信息工程学院）；School of Computer Science, Wuhan University（武汉大学计算机科学学院）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；Department of Computer Science, University of Macau（澳门大学计算机科学系）；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）
Infrared and Visible Image Fusion with Hierarchical Human Perception
School of Electronic Engineering, Xidian University（西安电子科技大学电子工程学院）；Key Laboratory of Image Cognition, Chongqing University of Posts and Telecommunications（重庆邮电大学图像认知重点实验室）
UHD Low-Light Image Enhancement via Real-Time Enhancement Methods with Clifford Information Fusion
Xi'an University of Electronic Science and Technology（西安电子科技大学）；National University of Defense Technology（国防科技大学）；School of Computer Science and Engineering, Nanjing University of Science and Technology（南京理工大学计算机科学与工程学院）；School of Information Science and Engineering, Shandong Normal University（山东师范大学信息科学与工程学院）；School of Computing and Artificial Intelligence, Shandong University of Finance and Economics（山东财经大学计算机科学与人工智能学院）；College of Information Science and Engineering, Shandong Agricultural University（山东农业大学信息科学与工程学院）；Xi'an University（西安大学）；Qilu University of Technology（齐鲁工业大学）
Multi-Branch Non-Homogeneous Image Dehazing via Concentration Partitioning and Image Fusion
School of Software and Internet of Things Engineering, Zhejiang Gongshang University（软件与物联网工程学院，浙江工商大学）；School of Computer Science and Technology, Tiangong University（计算机科学与技术学院，天工大学）
Baddi University of Emerging Sciences and Technology（巴迪新兴科学与技术大学）；Universidad Carlos III de Madrid（马德里卡洛斯三世大学）；Jaypee University of Information Technology（贾佩信息技术大学）
We develop a multiexposure image fusion method based on texture features, which exploits the edge preserving and intraregion smoothing property of nonlinear diffusion filters based on partial differential equations (PDE). With the captured multiexposure image series, we first decompose images into base layers and detail layers to extract sharp details and fine details, respectively. The magnitude of the gradient of the image intensity is utilized to encourage smoothness at homogeneous regions in preference to inhomogeneous regions. Then, we have considered texture features of the base layer to…
College of Computer Science & Technology, Qingdao University（青岛大学计算机科学与技术学院）；School of Computer Science & Engineering, South China University of Technology（华南理工大学计算机科学与工程学院）
We present a wavelet-based dual-stream network that addresses color cast and blurry details in underwater images. We handle these artifacts separately by decomposing an input image into multiple frequency bands using discrete wavelet transform, which generates the downsampled structure image and detail images. These sub-band images are used as input to our dual-stream network that incorporates two sub-networks: the multi-color space fusion network and the detail enhancement network. The multi-color space fusion network takes the decomposed structure image as input and estimates the color corre…
EvRWKV: A Continuous Interactive RWKV Framework for Effective Event-Guided Low-Light Image Enhancement
Anhui Provincial International Joint Research center for Advanced technology in Medical imaging（安徽省国际联合先进医学影像技术研究中心）；School of Artificial Intelligence（人工智能学院）；Anhui University（安徽大学）；State Key Laboratory of Opto-Electronic Information Acquisition and Protection Technology（光电信息采集与防护技术国家重点实验室）；Anhui Provincial Key Laboratory of Secure Artificial Intelligence（安徽省安全人工智能重点实验室）；Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ)（粤港澳大湾区人工智能与数字经济实验室（深圳））
Shanghai University（上海大学）；University of Leicester（莱斯特大学）
Low light very likely leads to the degradation of an image's quality and even causes visual task failures. Existing image enhancement technologies are prone to overenhancement, color distortion or time consumption, and their adaptability is fairly limited. Therefore, we propose a new single low-light image lightness enhancement method. First, an energy model is presented based on the analysis of membrane vibrations induced by photon stimulations. Then, based on the unique mathematical properties of the energy model and combined with the gamma correction model, a new global lightness enhancemen…
Southwestern University of Finance and Economics（西南财经大学）
Bike-sharing systems (BSSs) have become increasingly popular around the globe and have attracted a wide range of research interests. In this paper, the demand forecasting problem in BSSs is studied. Spatial and temporal features are critical for demand forecasting in BSSs, but it is challenging to extract spatiotemporal dynamics. Another challenge is to capture the relations between spatiotemporal dynamics and external factors, such as weather, day-of-week, and time-of-day. To address these challenges, we propose a multiple spatiotemporal fusion network named MSTF-Net. MSTF-Net consists of mul…
Institute of Cartography and Geo-informatics, Leibniz University Hannover, Germany（汉诺威莱布尼茨大学制图与地理信息研究所）
Sharing collective perception messages (CPM) between vehicles is investigated to decrease occlusions so as to improve the perception accuracy and safety of autonomous driving. However, highly accurate data sharing and low communication overhead is a big challenge for collective perception, especially when real-time communication is required among connected and automated vehicles. In this paper, we propose an efficient and effective keypoints-based deep feature fusion framework built on the 3D object detector PV-RCNN, called Fusion PV-RCNN (FPV-RCNN for short), for collective perception. We int…
Tianjin University（天津大学）；JD Finance America Corporation（京东金融美国公司）；University at Albany, State University of New York（纽约州立大学奥尔巴尼分校）
Drone equipped with cameras can dynamically track the target in the air from a broader view compared with static cameras or moving sensors over the ground. However, it is still challenging to accurately track the target using a single drone due to several factors such as appearance variations and severe occlusions. In this paper, we collect a new Multi-Drone single Object Tracking (MDOT) dataset that consists of 92 groups of video clips with 113,918 high resolution frames taken by two drones and 63 groups of video clips with 145,875 high resolution frames taken by three drones. Besides, two ev…
SIESEF-FusionNet: Spatial Inter-correlation Enhancement and Spatially-Embedded Feature Fusion Network for LiDAR Point Cloud Semantic Segmentation
Intelligent Autonomous Systems Laboratory, Shanghai University of Electric Power（上海电力大学智能自主系统实验室）；College of Literature, Science and the Arts, University of Michigan-Ann Arbor（密歇根大学安娜堡分校文学、科学与艺术学院）
Hikvision Research Institute（海康威视研究院）；Zhejiang University（浙江大学）；University of Science and Technology of China（中国科学技术大学）
One-shot neural architecture search (NAS) applies weight-sharing supernet to reduce the unaffordable computation overhead of automated architecture designing. However, the weight-sharing technique worsens the ranking consistency of performance due to the interferences between different candidate networks. To address this issue, we propose a candidates enhancement method and progressive training pipeline to improve the ranking correlation of supernet. Specifically, we carefully redesign the sub-networks in the supernet and map the original supernet to a new one of high capacity. In addition, we…
Ultra-high resolution image segmentation has raised increasing interests in recent years due to its realistic applications. In this paper, we innovate the widely used high-resolution image segmentation pipeline, in which an ultra-high resolution image is partitioned into regular patches for local segmentation and then the local results are merged into a high-resolution semantic mask. In particular, we introduce a novel locality-aware context fusion based segmentation model to process local patches, where the relevance between local patch and its various contexts are jointly and complementarily…
Perceptual Region-Driven Infrared-Visible Co-Fusion for Extreme Scene Enhancement
College of Aerospace Science and Engineering, National University of Defense Technology（航天科学与工程学院，国防科技大学）；Hunan Provincial Key Laboratory of Image Measurement and Vision Navigation（湖南省图像测量与视觉导航重点实验室）；Beijing Institute of Tracking and Telecommunication Technology（北京跟踪与电信技术研究所）；National Key Laboratory of Space Integrated Information System（空间一体化信息系统国家重点实验室）
Video caption refers to generating a descriptive sentence for a specific short video clip automatically, which has achieved remarkable success recently. However, most of the existing methods focus more on visual information while ignoring the synchronized audio cues. We propose three multimodal deep fusion strategies to maximize the benefits of visual-audio resonance information. The first one explores the impact on cross-modalities feature fusion from low to high order. The second establishes the visual-audio short-term dependency by sharing weights of corresponding front-end networks. The th…
Images captured underwater often suffer from suboptimal illumination settings that can hide important visual features, reducing their quality. We present a novel single-image low-light underwater image enhancer, L^2UWE, that builds on our observation that an efficient model of atmospheric lighting can be derived from local contrast information. We create two distinct models and generate two enhanced images from them: one that highlights finer details, the other focused on darkness removal. A multi-scale fusion process is employed to combine these images while emphasizing regions of higher lumi…
A novel network for enhancement to underwater images is proposed in this paper. It contains a Reinforcement Fusion Module for Haar wavelet images (RFM-Haar) based on Reinforcement Fusion Unit (RFU), which is used to fuse an original image and some important information within it. Fusion is achieved for better enhancement. As this network make "Haar Images into Fusion Images", it is called HIFI-Net. The experimental results show the proposed HIFI-Net performs best among many state-of-the-art methods on three datasets at three normal metrics and a new metric.
EeveeDark: A Binary Neural Framework for Low-Light Video Enhancement via Event-Guided Sensor-Level Fusion
Department of Computer Engineering, Hacettepe University（哈杰泰佩大学计算机工程系）；HAVELSAN Inc.（哈韦尔桑公司）；Koc University Is Bank AI Center（科克大学伊希银行人工智能中心）；Department of Computer Engineering, Koc University（科克大学计算机工程系）
Context Patch Fusion With Class Token Enhancement for Weakly Supervised Semantic Segmentation
School of Cyber Science and Engineering, Wuxi University（网络安全科学与工程学院，无锡大学）；School of Informatics, Xiamen University（信息学院，厦门大学）；School of Computer Science, University of Liverpool（计算机科学学院，利物浦大学）
Zero-Shot Video Restoration and Enhancement with Assistance of Video Diffusion Models
School of Electrical and Information Engineering, Tianjin University（电子信息工程学院，天津大学）；Computer Vision and Pattern Recognition Laboratory, School of Engineering Science, Lappeenranta-Lahti University of Technology LUT（计算机视觉与模式识别实验室，工程科学学院，拉佩兰塔-拉赫蒂技术大学LUT）
Southeast University（东南大学）；Shanghai United-imaging Healthcare Co., Ltd（上海联影医疗科技股份有限公司）；University of Birmingham（伯明翰大学）
In order to improve speckle noise denoising of block matching 3d filtering (BM3D) method, an image frequency-domain multi-layer fusion enhancement method (MLFE-BM3D) based on nonsubsampled contourlet transform (NSCT) has been proposed. The method designs a NSCT hard threshold denoising enhancement to preprocess the image, then uses fusion enhancement in NSCT domain to fuse the preliminary estimation results of images before and after the NSCT hard threshold denoising, finally, BM3D denoising is carried out with the fused image to obtain the final denoising result. Experiments on natural images…
Kunming University of Science and Technology（昆明理工大学）
Multispectral pedestrian detection is a technology designed to detect and locate pedestrians in Color and Thermal images, which has been widely used in automatic driving, video surveillance, etc. So far most available multispectral pedestrian detection algorithms only achieved limited success in pedestrian detection because of the lacking take into account the confusion of pedestrian information and background noise in Color and Thermal images. Here we propose a multispectral pedestrian detection algorithm, which mainly consists of a cascaded information enhancement module and a cross-modal at…
DEPFusion: Dual-Domain Enhancement and Priority-Guided Mamba Fusion for UAV Multispectral Object Detection
School of Information Engineering, Guangdong University of Technology（广东工业大学信息工程学院）；School of Electronic and Information Engineering, South China University of Technology（华南理工大学电子信息学院）；School of Artificial Intelligence, XiDian University（西安电子科技大学人工智能学院）
