Limited by the cost and technology, the resolution of depth map collected by depth camera is often lower than that of its associated RGB camera. Although there have been many researches on RGB image super-resolution (SR), a major problem with depth map super-resolution is that there will be obvious jagged edges and excessive loss of details. To tackle these difficulties, in this work, we propose a multi-scale progressive fusion network for depth map SR, which possess an asymptotic structure to integrate hierarchical features in different domains. Given a low-resolution (LR) depth map and its associated high-resolution (HR) color image, We utilize two different branches to achieve multi-scale feature learning. Next, we propose a step-wise fusion strategy to restore the HR depth map. Finally, a multi-dimensional loss is introduced to constrain clear boundaries and details. Extensive experiments show that our proposed method produces improved results against state-of-the-art methods both qualitatively and quantitatively.
Jiangxi Normal University（江西师范大学）；East China Normal University（华东师范大学）
Single image dehazing is a challenging ill-posed restoration problem. Various prior-based and learning-based methods have been proposed. Most of them follow a classic atmospheric scattering model which is an elegant simplified physical model based on the assumption of single-scattering and homogeneous atmospheric medium. The formulation of haze in realistic environment is more complicated. In this paper, we propose to take its essential mechanism as "black box", and focus on learning an input-adaptive trainable end-to-end dehazing model. An U-Net like encoder-decoder deep network via progressi…
EPOFusion: Exposure aware Progressive Optimization Method for Infrared and Visible Image Fusion
College of Information Engineering, Zhejiang University of Technology（浙江工业大学信息工程学院）；College of Information Science and Technology, Zhejiang Shuren University（浙江树人大学信息科技学院）；Department of Electrical and Electronic Engineering, The University of Hong Kong（香港大学电机电子工程系）
Hong Kong Baptist University（香港浸会大学）；Institute of High Performance Computing, A*STAR（新加坡科技研究局高性能计算研究院）；vivo Mobile Communication Co. Ltd（维沃移动通信有限公司）；City University of Hong Kong（香港城市大学）；Nanyang Technological University（南洋理工大学）
High Dynamic Range (HDR) imaging via multi-exposure fusion is an important task for most modern imaging platforms. In spite of recent developments in both hardware and algorithm innovations, challenges remain over content association ambiguities caused by saturation, motion, and various artifacts introduced during multi-exposure fusion such as ghosting, noise, and blur. In this work, we propose an Attention-guided Progressive Neural Texture Fusion (APNT-Fusion) HDR restoration model which aims to address these issues within one framework. An efficient two-stream structure is proposed which sep…
P2Fusion: Prompt-based Progressive Infrared-Visible Image Fusion via Dual-Prior Distillation
Northwestern Polytechnical University（西北工业大学）；Hefei University of Technology（合肥工业大学）；Rocket Force University of Engineering（火箭军工程大学）；Chongqing University of Posts and Telecommunications（重庆邮电大学）
Enhancing Infrared Vision: Progressive Prompt Fusion Network and Benchmark
School of Software Engineering, Dalian University of Technology（大连理工大学软件学院）；Information Science and Technology College, Dalian Martime University（大连海事大学信息科学与技术学院）
Fusing a low-resolution hyperspectral image (HSI) and a high-resolution multispectral image (MSI) of the same scene leads to a super-resolution image (SRI), which is information rich spatially and spectrally. In this paper, we super-resolve the HSI using the graph Laplacian defined on the MSI. Unlike many existing works, we don't assume prior knowledge about the spatial degradation from SRI to HSI, nor a perfectly aligned HSI and MSI pair. Our algorithm progressively alternates between finding the blur kernel and fusing HSI with MSI, generating accurate estimations of the blur kernel and the S…
LoopLUT: 3D Lookup Tables with Progressive Region Refinement for Real-Time 4K Image Enhancement
Universiti Sains Malaysia（马来西亚理科大学）；National University of Defense Technology（国防科技大学）；Sun Yat-sen University（中山大学）；Shandong Normal University（山东师范大学）；Shandong University of Finance and Economics（山东财经大学）
Indian Institute of Technology Kharagpur（印度理工学院卡拉格普尔分校）
In this paper, we propose a novel method for the challenging problem of guided depth map super-resolution, called PAGNet. It is based on residual dense networks and involves the attention mechanism to suppress the texture copying problem arises due to improper guidance by RGB images. The attention module mainly involves providing the spatial attention to guidance image based on the depth features. We evaluate the proposed trained models on test dataset and provide comparisons with the state-of-the-art depth super-resolution methods.
Face Super-Resolution (SR) is a subfield of the SR domain that specifically targets the reconstruction of face images. The main challenge of face SR is to restore essential facial features without distortion. We propose a novel face SR method that generates photo-realistic 8x super-resolved face images with fully retained facial details. To that end, we adopt a progressive training method, which allows stable training by splitting the network into successive steps, each producing output with a progressively higher resolution. We also propose a novel facial attention loss and apply it at each s…
Recent deep learning approaches to single image super-resolution have achieved impressive results in terms of traditional error measures and perceptual quality. However, in each case it remains challenging to achieve high quality results for large upsampling factors. To this end, we propose a method (ProSR) that is progressive both in architecture and training: the network upsamples an image in intermediate steps, while the learning process is organized from easy to hard, as is done in curriculum learning. To obtain more photorealistic results, we design a generative adversarial network (GAN),…
Sri Jayachamrajendra College of Engineering（斯里贾亚查姆拉金德拉工程学院）
Due to the factors like processing power limitations and channel capabilities images are often down sampled and transmitted at low bit rates resulting in a low resolution compressed image. High resolution images can be reconstructed from several blurred, noisy and down sampled low resolution images using a computational process know as super resolution reconstruction. Super-resolution is the process of combining multiple aliased low-quality images to produce a high resolution, high-quality image. The problem of recovering a high resolution image progressively from a sequence of low resolution …
University of Maryland（马里兰大学）；Academia Sinica（中央研究院）；Johns Hopkins University（约翰斯·霍普金斯大学）
Face super-resolution is a challenging and highly ill-posed problem since a low-resolution (LR) face image may correspond to multiple high-resolution (HR) ones during the hallucination process and cause a dramatic identity change for the final super-resolved results. Thus, to address this problem, we propose an end-to-end progressive learning framework incorporating facial attributes and enforcing additional supervision from multi-scale discriminators. By incorporating facial attributes into the learning process and progressively resolving the facial image, the mapping between LR and HR images…
Tianjin University（天津大学）；College of Intelligence and Computing, Tianjin University（天津大学智能与计算学部）
The face super-resolution (FSR) task is to reconstruct high-resolution face images from low-resolution inputs. Recent works have achieved success on this task by utilizing facial priors such as facial landmarks. Most existing methods pay more attention to global shape and structure information, but less to local texture information, which makes them cannot recover local details well. In this paper, we propose a novel recurrent convolutional network based framework for face super-resolution, which progressively introduces both global shape and local texture information. We take full advantage o…
Mask-adaptive Gated Convolution and Bi-directional Progressive Fusion Network for Depth Completion
College of Information Science and Engineering, Northeastern University（东北大学信息科学与工程学院）；Foshan Graduate School of Innovation, Northeastern University（东北大学佛山创新研究生院）；National Frontiers Science Center for Industrial Intelligence and Systems Optimization, Northeastern University（东北大学工业智能与系统优化国家前沿科学中心）
We propose a new approach for the image super-resolution (SR) task that progressively restores a high-resolution (HR) image from an input low-resolution (LR) image on the basis of a neural ordinary differential equation. In particular, we newly formulate the SR problem as an initial value problem, where the initial value is the input LR image. Unlike conventional progressive SR methods that perform gradual updates using straightforward iterative mechanisms, our SR process is formulated in a concrete manner based on explicit modeling with a much clearer understanding. Our method can be easily i…
Imperial College London（伦敦帝国学院）；University of Liverpool（利物浦大学）
Salient object detection(SOD) aims at locating the most significant object within a given image. In recent years, great progress has been made in applying SOD on many vision tasks. The depth map could provide additional spatial prior and boundary cues to boost the performance. Combining the depth information with image data obtained from standard visual cameras has been widely used in recent SOD works, however, introducing depth information in a suboptimal fusion strategy may have negative influence in the performance of SOD. In this paper, we discuss about the advantages of the so-called prog…
Single Image Super-Resolution (SISR) aims to improve resolution of small-size low-quality image from a single one. With popularity of consumer electronics in our daily life, this topic has become more and more attractive. In this paper, we argue that the curse of dimensionality is the underlying reason of limiting the performance of state-of-the-art algorithms. To address this issue, we propose Progressive Adversarial Network (PAN) that is capable of coping with this difficulty for domain-specific image super-resolution. The key principle of PAN is that we do not apply any distance-based recon…
Deepwise AI Lab（深睿人工智能实验室）；The University of Hong Kong（香港大学）；Sun Yat-sen University（中山大学）
Recently 3D volumetric organ segmentation attracts much research interest in medical image analysis due to its significance in computer aided diagnosis. This paper aims to address the pancreas segmentation task in 3D computed tomography volumes. We propose a novel end-to-end network, Globally Guided Progressive Fusion Network, as an effective and efficient solution to volumetric segmentation, which involves both global features and complicated 3D geometric information. A progressive fusion network is devised to extract 3D information from a moderate number of neighboring slices and predict a p…
University of Trento（特伦托大学）；University of Oxford（牛津大学）；Fondazione Bruno Kessler（布鲁诺·凯塞勒基金会）；Huawei Technologies Ireland（华为技术爱尔兰）
Recent deep monocular depth estimation approaches based on supervised regression have achieved remarkable performance. However, they require costly ground truth annotations during training. To cope with this issue, in this paper we present a novel unsupervised deep learning approach for predicting depth maps. We introduce a new network architecture, named Progressive Fusion Network (PFN), that is specifically designed for binocular stereo depth estimation. This network is based on a multi-scale refinement strategy that combines the information provided by both stereo views. In addition, we pro…
DepMamba: Progressive Fusion Mamba for Multimodal Depression Detection
Institute of Science and Technology for Brain-inspired Intelligence, Fudan University（复旦大学脑智能科学与技术研究院）；School of Computer Science, Fudan University（复旦大学计算机科学技术学院）
Tohoku University（东北大学）；The Hong Kong Polytechnic University（香港理工大学）；RIKEN Center for AIP（理化学研究所革新智能统合研究中心）
This paper considers the problem of generating an HDR image of a scene from its LDR images. Recent studies employ deep learning and solve the problem in an end-to-end fashion, leading to significant performance improvements. However, it is still hard to generate a good quality image from LDR images of a dynamic scene captured by a hand-held camera, e.g., occlusion due to the large motion of foreground objects, causing ghosting artifacts. The key to success relies on how well we can fuse the input images in their feature space, where we wish to remove the factors leading to low-quality image ge…
DCFG: Diverse Cross-Channel Fine-Grained Feature Learning and Progressive Fusion Siamese Tracker for Thermal Infrared Target Tracking
College of Computer and Information Technology, China Three Gorges University（中国三峡大学计算机与信息学院）；Hubei Province Engineering Technology Research Center for Construction Quality Testing Equipment, China Three Gorges University（中国三峡大学湖北省建筑质量检测装备工程技术研究中心）；Wuhan College（武汉学院）
MPFNet: A Multi-Prior Fusion Network with a Progressive Training Strategy for Micro-Expression Recognition
Defense Innovation Institute, Academy of Military Sciences (AMS) and Intelligent Game and Decision Laboratory（国防科技创新院、军事科学研究院（AMS）和智能游戏与决策实验室）；School of Computer Science and Technology, Dalian University of Technology（计算机科学与技术学院、大连理工大学）
Dalian University of Technology（大连理工大学）；Tsinghua University（清华大学）
Glass is very common in the real world. Influenced by the uncertainty about the glass region and the varying complex scenes behind the glass, the existence of glass poses severe challenges to many computer vision tasks, making glass segmentation as an important computer vision task. Glass does not have its own visual appearances but only transmit/reflect the appearances of its surroundings, making it fundamentally different from other common objects. To address such a challenging task, existing methods typically explore and combine useful cues from different levels of features in the deep netw…
HAPNet: Toward Superior RGB-Thermal Scene Parsing via Hybrid, Asymmetric, and Progressive Heterogeneous Feature Fusion
College of Electronic and Information Engineering, Tongji University（同济大学电子与信息工程学院）；Department of Computer Science and Engineering, Hong Kong University of Science and Technology（香港科技大学计算机科学与工程系）；MSU-BIT-SMBU Joint Research Center of Applied Mathematics, Shenzhen MSU-BIT University（深圳MSU-BIT大学应用数学联合研究中心）；Qingdao Innovation and Development Base, Harbin Institute of Technology (Weihai)（哈尔滨工业大学（威海）青岛创新与发展基地）；Department of Control Science and Engineering, Harbin Institute of Technology（哈尔滨工业大学控制科学与工程系）；Harbin Institute of Technology at Weihai（哈尔滨工业大学（威海））；Suzhou Research Institute, Harbin Institute of Technology（哈尔滨工业大学苏州研究所）；Faculty of Computational Mathematics and Cybernetics, Lomonosov Moscow State University（莫斯科罗蒙诺夫莫斯科国立大学计算数学与自动化系）
Wuhan University（武汉大学）；University of North Carolina at Charlotte（北卡罗来纳大学夏洛特分校）；King’s College London（伦敦国王学院）；Harbin Institute of Technology（哈尔滨工业大学）
Rain streaks in the air appear in various blurring degrees and resolutions due to different distances from their positions to the camera. Similar rain patterns are visible in a rain image as well as its multi-scale (or multi-resolution) versions, which makes it possible to exploit such complementary information for rain streak representation. In this work, we explore the multi-scale collaborative representation for rain streaks from the perspective of input image scales and hierarchical deep features in a unified framework, termed multi-scale progressive fusion network (MSPFN) for single image…
Light-A-Video: Training-free Video Relighting via Progressive Light Fusion
Shanghai Jiao Tong University（上海交通大学）；University of Science and Technology of China（中国科学技术大学）；The Chinese University of Hong Kong（香港中文大学）；Hong Kong University of Science and Technology（香港科技大学）；Stanford University（斯坦福大学）；Shanghai AI Laboratory（上海人工智能实验室）
Britton Chance Center for Biomedical Photonics（布里顿·钱斯生物医学光子学研究中心）；Wuhan National Laboratory for Optoelectronics（武汉光电国家研究中心）；MoE Key Laboratory for Biomedical Photonics（教育部生物医学光子学重点实验室）；Collaborative Innovation Center for Biomedical Engineering（生物医学工程协同创新中心）；School of Engineering Sciences, Huazhong University of Science and Technology（华中科技大学工程科学学院）
Registration of brain MRI images requires to solve a deformation field, which is extremely difficult in aligning intricate brain tissues, e.g., subcortical nuclei, etc. Existing efforts resort to decomposing the target deformation field into intermediate sub-fields with either tiny motions, i.e., progressive registration stage by stage, or lower resolutions, i.e., coarse-to-fine estimation of the full-size deformation field. In this paper, we argue that those efforts are not mutually exclusive, and propose a unified framework for robust brain MRI registration in both progressive and coarse-to-…
PCaM: A Progressive Focus Attention-Based Information Fusion Method for Improving Vision Transformer Domain Adaptation
Westlake Institute for Advanced Studies（西湖研究学院）；Centre for Artificial Intelligence（人工智能中心）；State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS)（多模态人工智能系统国家重点实验室）；Institute of Automation（自动化研究所）；Chinese Academy of Sciences (CASIA)（中国科学院）；School of Artificial Intelligence（人工智能学院）；University of Chinese Academy of Sciences (UCAS)（中国科学院大学）；Alibaba Group（阿里巴巴集团）；School of Computer Science（计算机学院）；Engineering, the Faculty of Innovation Engineering, Macau University of Science（工程学院，创新工程 faculty，澳门科学大学）；Microsoft（微软公司）
Rethinking Air-Ground Collaboration: A Progressive Cross-Task Benchmark and Socialized Learning Framework
School of Automation, Southeast University（东南大学自动化学院）；School of Computer Science and Engineering, University of New South Wales（新南威尔士大学计算机科学与工程学院）；School of Sports Training, Tianjin University of Sport（天津体育学院运动训练学院）；Faculty of Information Engineering and Automation, Kunming University of Science and Technology（昆明理工大学信息工程与自动化学院）；School of Artificial Intelligence, Tianjin University（天津大学人工智能学院）；School of Artificial Intelligence, Hebei University of Technology（河北工业大学人工智能学院）
In this paper, we aim to develop an efficient and compact deep network for RGB-D salient object detection, where the depth image provides complementary information to boost performance in complex scenarios. Starting from a coarse initial prediction by a multi-scale residual block, we propose a progressively guided alternate refinement network to refine it. Instead of using ImageNet pre-trained backbone network, we first construct a lightweight depth stream by learning from scratch, which can extract complementary features more efficiently with less redundancy. Then, different from the existing…
E2S-Pruner: Progressive Two-Stage Evidence Fusion for Visual Token Pruning in Vision-Language Models
Digital Environment Research Institute (DERI), Queen Mary University of London（伦敦大学玛丽女王学院数字环境研究所（DERI））；Shanghai Institute of Artificial Intelligence for Education, East China Normal University（华东师范大学上海人工智能教育研究院）；National Institute of Education, Nanyang Technological University（南洋理工大学国家教育学院）；School of Computer Science and Engineering, Jiangsu University of Science and Technology（江苏科技大学计算机科学与工程学院）
We propose a compact and effective framework to fuse multimodal features at multiple layers in a single network. The framework consists of two innovative fusion schemes. Firstly, unlike existing multimodal methods that necessitate individual encoders for different modalities, we verify that multimodal features can be learnt within a shared single network by merely maintaining modality-specific batch normalization layers in the encoder, which also enables implicit fusion via joint feature representation learning. Secondly, we propose a bidirectional multi-layer fusion scheme, where multimodal f…
Most of the existing semantic segmentation approaches with image-level class labels as supervision, highly rely on the initial class activation map (CAM) generated from the standard classification network. In this paper, a novel "Progressive Patch Learning" approach is proposed to improve the local details extraction of the classification, producing the CAM better covering the whole object rather than only the most discriminative regions as in CAMs obtained in conventional classification models. "Patch Learning" destructs the feature maps into patches and independently processes each local pat…
Recently, referring image segmentation has aroused widespread interest. Previous methods perform the multi-modal fusion between language and vision at the decoding side of the network. And, linguistic feature interacts with visual feature of each scale separately, which ignores the continuous guidance of language to multi-scale visual features. In this work, we propose an encoder fusion network (EFN), which transforms the visual encoder into a multi-modal feature learning network, and uses language to refine the multi-modal features progressively. Moreover, a co-attention mechanism is embedded…
RGB-D semantic segmentation methods conventionally use two independent encoders to extract features from the RGB and depth data. However, there lacks an effective fusion mechanism to bridge the encoders, for the purpose of fully exploiting the complementary information from multiple modalities. This paper proposes a novel bottom-up interactive fusion structure to model the interdependencies between the encoders. The structure introduces an interaction stream to interconnect the encoders. The interaction stream not only progressively aggregates modality-specific features from the encoders but a…
Chongqing University（重庆大学）；Huazhong University of Science and Technology（华中科技大学）
The existing crowd counting methods usually adopted attention mechanism to tackle background noise, or applied multi-level features or multi-scales context fusion to tackle scale variation. However, these approaches deal with these two problems separately. In this paper, we propose a Hybrid Attention Network (HAN) by employing Progressive Embedding Scale-context (PES) information, which enables the network to simultaneously suppress noise and adapt head scale variation. We build the hybrid attention mechanism through paralleling spatial attention and channel attention module, which makes the n…
STAR-Pro: Stage-Wise Token Adaptive Reduction with Progressive Refinement for Efficient Large Vision-Language Models
Peking University（北京大学）；Nanyang Technological University, Singapore（南洋理工大学）；University of Electronic Science and Technology of China（电子科技大学）；University of Michigan, Ann Arbor（密歇根大学安娜堡分校）；De Artificial Intelligence Lab（De人工智能实验室）
