Recently, it has been demonstrated that deep neural networks can significantly improve the performance of single image super-resolution (SISR). Numerous studies have concentrated on raising the quantitative quality of super-resolved (SR) images. However, these methods that target PSNR maximization usually produce blurred images at large upscaling factor. The introduction of generative adversarial networks (GANs) can mitigate this issue and show impressive results with synthetic high-frequency textures. Nevertheless, these GAN-based approaches always have a tendency to add fake textures and even artifacts to make the SR image of visually higher-resolution. In this paper, we propose a novel perceptual image super-resolution method that progressively generates visually high-quality results by constructing a stage-wise network. Specifically, the first phase concentrates on minimizing pixel-wise error, and the second stage utilizes the features extracted by the previous stage to pursue results with better structural retention. The final stage employs fine structure features distilled by the second phase to produce more realistic results. In this way, we can maintain the pixel, and structural level information in the perceptual image as much as possible. It is useful to note that the proposed method can build three types of images in a feed-forward process. Also, we explore a new generator that adopts multi-scale hierarchical features fusion. Extensive experiments on benchmark datasets show that our approach is superior to the state-of-the-art methods. Code is available at https://github.com/Zheng222/PPON.
Recent studies of deep learning based stereo image super-resolution (StereoSR) have promoted the development of StereoSR. However, existing StereoSR models mainly concentrate on improving quantitative evaluation metrics and neglect the visual quality of super-resolved stereo images. To improve the perceptual performance, this paper proposes the first perception-oriented stereo image super-resolution approach by exploiting the feedback, provided by the evaluation on the perceptual quality of StereoSR results. To provide accurate guidance for the StereoSR model, we develop the first special ster…
University of Science and Technology of China（中国科学技术大学）；Shandong University（山东大学）；The University of Sydney（悉尼大学）
Contrast enhancement and noise removal are coupled problems for low-light image enhancement. The existing Retinex based methods do not take the coupling relation into consideration, resulting in under or over-smoothing of the enhanced images. To address this issue, this paper presents a novel progressive Retinex framework, in which illumination and noise of low-light image are perceived in a mutually reinforced manner, leading to noise reduction low-light enhancement results. Specifically, two fully pointwise convolutional neural networks are devised to model the statistical regularities of am…
Perception-oriented Bidirectional Attention Network for Image Super-resolution Quality Assessment
School of Mathematical Sciences, Beihang University（北京航空航天大学数学学院）；School of Computer Science and Informatics, Cardiff University（卡迪夫大学计算机科学与信息学院）；School of Biomedical Engineering, Shenzhen University（深圳大学生物医学工程学院）；CAS Key Laboratory of Technology in Geo-Spatial Information Processing and Application System, University of Science and Technology of China（中国科学技术大学地球空间信息处理与应用系统国家重点实验室）；School of Information Science and Engineering, Ningbo University（宁波大学信息科学与工程学院）；School of Artificial Intelligence, Dalian University of Technology（大连理工大学人工智能学院）
Seoul National University（首尔国立大学）；Samsung Electronics（三星电子）
Single-image super-resolution (SISR) networks trained with perceptual and adversarial losses provide high-contrast outputs compared to those of networks trained with distortion-oriented losses, such as L1 or L2. However, it has been shown that using a single perceptual loss is insufficient for accurately restoring locally varying diverse shapes in images, often generating undesirable artifacts or unnatural details. For this reason, combinations of various losses, such as perceptual, adversarial, and distortion losses, have been attempted, yet it remains challenging to find optimal combinations…
University of Science and Technology of China（中国科学技术大学）
Deep neural networks (DNNs) have shown great potential in non-reference image quality assessment (NR-IQA). However, the annotation of NR-IQA is labor-intensive and time-consuming, which severely limits their application especially for authentic images. To relieve the dependence on quality annotation, some works have applied unsupervised domain adaptation (UDA) to NR-IQA. However, the above methods ignore that the alignment space used in classification is sub-optimal, since the space is not elaborately designed for perception. To solve this challenge, we propose an effective perception-oriented…
Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院）
Deep Convolutional Neural Network (CNN) features have been demonstrated to be effective perceptual quality features. The perceptual loss, based on feature maps of pre-trained CNN's has proven to be remarkably effective for CNN based perceptual image restoration problems. In this work, taking inspiration from the the Human Visual System (HVS) and visual perception, we propose a spatial attention mechanism based on the dependency human contrast sensitivity on spatial frequency. We identify regions in input images, based on the underlying spatial frequency, which are not generally well reconstruc…
Dual-Latent Collaborative Decoding for Fidelity-Perception Balanced Image Compression
School of Information and Communication Engineering and the State Key Laboratory of Media Convergence and Communication, Communication University of China（信息与通信工程学院和媒体融合与通信国家重点实验室，中国通信大学）；School of Information Science and Electronic Engineering, Shanghai Jiao Tong University（信息科学与电子工程学院，上海交通大学）；Department of Computer Science, City University of Hong Kong（计算机科学系，香港城市大学）；State Key Laboratory of Multimedia information Processing, School of Computer Science, Peking University（多媒体信息处理国家重点实验室，计算机科学学院，北京大学）
EPOFusion: Exposure aware Progressive Optimization Method for Infrared and Visible Image Fusion
College of Information Engineering, Zhejiang University of Technology（浙江工业大学信息工程学院）；College of Information Science and Technology, Zhejiang Shuren University（浙江树人大学信息科技学院）；Department of Electrical and Electronic Engineering, The University of Hong Kong（香港大学电机电子工程系）
P2Fusion: Prompt-based Progressive Infrared-Visible Image Fusion via Dual-Prior Distillation
Northwestern Polytechnical University（西北工业大学）；Hefei University of Technology（合肥工业大学）；Rocket Force University of Engineering（火箭军工程大学）；Chongqing University of Posts and Telecommunications（重庆邮电大学）
Qingdao University（青岛大学）；New York University（纽约大学）；Shandong University（山东大学）；De Montfort University（德蒙福特大学）
With the wide applications of colored point cloud in many fields, point cloud perceptual quality assessment plays a vital role in the visual communication systems owing to the existence of quality degradations introduced in various stages. However, the existing point cloud quality assessments ignore the mechanism of human visual system (HVS) which has an important impact on the accuracy of the perceptual quality assessment. In this paper, a progressive knowledge transfer based on human visual perception mechanism for perceptual quality assessment of point clouds (PKT-PCQA) is proposed. The PKT…
We describe our solution for the PIRM Super-Resolution Challenge 2018 where we achieved the 2nd best perceptual quality for average RMSE<=16, 5th best for RMSE<=12.5, and 7th best for RMSE<=11.5. We modify a recently proposed Multi-Grid Back-Projection (MGBP) architecture to work as a generative system with an input parameter that can control the amount of artificial details in the output. We propose a discriminator for adversarial training with the following novel properties: it is multi-scale that resembles a progressive-GAN; it is recursive that balances the architecture of the generator; a…
One at a Time: Progressive Multi-step Volumetric Probability Learning for Reliable 3D Scene Perception
Shanghai Jiao Tong University（上海交通大学）；Ningbo Institute of Digital Twin, Eastern Institute of Technology（宁波数字孪生研究院，东方理工）；Tokyo Institute of Technology（东京工业大学）；PhiGent Robotics（PhiGent机器人公司）
Progressive Self-Distillation for Ground-to-Aerial Perception Knowledge Transfer
Shenzhen Institute of Artificial Intelligence and Robotics for Society（深圳市人工智能与机器人研究院）；South China Normal University（华南师范大学）；Samsung Research（三星研究院）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））
Wuhan University of Technology（武汉理工大学）；Hubei Key Laboratory of Broadband Wireless Communication and Sensor Networks（湖北省宽带无线通信与传感器网络重点实验室）；School of Civil Engineering & Architecture（土木工程与建筑学院）；School of Information Engineering（信息工程学院）；State Key Laboratory of Magnetic Resonance and Atomic Molecular Phys（磁共振与原子分子物理国家重点实验室）
The presence of residual and dense neural networks which greatly promotes the development of image Super-Resolution(SR) have witnessed a lot of impressive results. Depending on our observation, although more layers and connections could always improve performance, the increase of model parameters is not conducive to launch application of SR algorithms. Furthermore, algorithms supervised by L1/L2 loss can achieve considerable performance on traditional metrics such as PSNR and SSIM, yet resulting in blurry and over-smoothed outputs without sufficient high-frequency details, namely low perceptua…
Progressive Bird's Eye View Perception for Safety-Critical Autonomous Driving: A Comprehensive Survey
State Key Laboratory of Robotics and System（机器人系统国家重点实验室）；Harbin Institute of Technology（哈尔滨工业大学）；State Key Laboratory of Intelligent Green Vehicle and Mobility（智能绿色车辆与移动性国家重点实验室）；Tsinghua University（清华大学）；the School of Mechanical and Aerospace Engineering（机械与航空航天工程学院）；Nanyang Technological University（南洋理工大学）；Beijing Key Laboratory of Traffic Data Mining and Embodied Intelligence（北京交通数据挖掘与具身智能重点实验室）；Beijing Jiaotong University（北京交通大学）；the Institute for Infocomm Research（信息通信研究所）；A*STAR；the Engineering Cluster（工程集群）；the Singapore Institute of Technology（新加坡理工学院）
MMUSV-Sim: A Perception-Oriented Simulation and Data-Generation Platform for Multi-USV Cooperative Perception
School of Intelligent Systems Engineering, Sun Yat-sen University（中山大学智能工程学院）；School of Intelligent Imagery Engineering, Beijing Film Academy（北京电影学院智能影像工程学院）
CORP: A Multi-Modal Dataset for Campus-Oriented Roadside Perception Tasks
Institute of Artificial Intelligence, Hefei Comprehensive National Science Center（合肥综合国家科学中心人工智能研究院）；University of Science and Technology of China（中国科学技术大学）
Rethinking Air-Ground Collaboration: A Progressive Cross-Task Benchmark and Socialized Learning Framework
School of Automation, Southeast University（东南大学自动化学院）；School of Computer Science and Engineering, University of New South Wales（新南威尔士大学计算机科学与工程学院）；School of Sports Training, Tianjin University of Sport（天津体育学院运动训练学院）；Faculty of Information Engineering and Automation, Kunming University of Science and Technology（昆明理工大学信息工程与自动化学院）；School of Artificial Intelligence, Tianjin University（天津大学人工智能学院）；School of Artificial Intelligence, Hebei University of Technology（河北工业大学人工智能学院）
Perception-Oriented Video Frame Interpolation via Asymmetric Blending
Shanghai Jiao Tong University（上海交通大学）；Kuaishou Technology（快手科技）；SeeKoo（视库）；University of Electronic Science and Technology of China（电子科技大学）；Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）
In this paper, we are concerned with the detection of progressive dynamic saliency from video sequences. More precisely, we are interested in saliency related to motion and likely to appear progressively over time. It can be relevant to trigger alarms, to dedicate additional processing or to detect specific events. Trajectories represent the best way to support progressive dynamic saliency detection. Accordingly, we will talk about trajectory saliency. A trajectory will be qualified as salient if it deviates from normal trajectories that share a common motion pattern related to a given context…
University of Michigan（密歇根大学）；Robotics Institute（机器人研究所）
In order to enable robust operation in unstructured environments, robots should be able to generalize manipulation actions to novel object instances. For example, to pour and serve a drink, a robot should be able to recognize novel containers which afford the task. Most importantly, robots should be able to manipulate these novel containers to fulfill the task. To achieve this, we aim to provide robust and generalized perception of object affordances and their associated manipulation poses for reliable manipulation. In this work, we combine the notions of affordance and category-level pose, an…
Safety-Oriented Trajectory Ranking from BEV Perception Outputs
University of the Bundeswehr Munich（德国联邦国防军慕尼黑大学）；Ludwig-Maximilians-Universität München（慕尼黑大学）；Research Institute CODE, University of the Bundeswehr Munich（德国联邦国防军慕尼黑大学CODE研究院）
Self-supervised Multi-task Learning Framework for Safety and Health-Oriented Connected Driving Environment Perception using Onboard Camera
The Hong Kong Polytechnic University（香港理工大学）；The University of Hong Kong（香港大学）；The Hong Kong Polytechnic University Shenzhen Research Institute（香港理工大学深圳研究院）
Image manipulation localization aims at distinguishing forged regions from the whole test image. Although many outstanding prior arts have been proposed for this task, there are still two issues that need to be further studied: 1) how to fuse diverse types of features with forgery clues; 2) how to progressively integrate multistage features for better localization performance. In this paper, we propose a tripartite progressive integration network (TriPINet) for end-to-end image manipulation localization. First, we extract both visual perception information, e.g., RGB input images, and visual i…
We propose a novel tree-like curvilinear structure reconstruction algorithm based on supervised learning and graph theory. In this work we analyze image patches to obtain the local major orientations and the rankings that correspond to the curvilinear structure. To extract local curvilinear features, we compute oriented gradient information using steerable filters. We then employ Structured Support Vector Machine for ordinal regression of the input image patches, where the ordering is determined by shape similarity to latent curvilinear structure. Finally, we progressively reconstruct the curv…
Achelous++: Power-Oriented Water-Surface Panoptic Perception Framework on Edge Devices based on Vision-Radar Fusion and Pruning of Heterogeneous Modalities
University of Liverpool（利物浦大学）；Xi’an Jiaotong-Liverpool University（西交利物浦大学）；JITRI（江苏省产业技术研究院）；Nantong University（南通大学）
ETH Zürich（苏黎世联邦理工学院）；University of Wurzburg（维尔茨堡大学）；KU Leuven（鲁汶大学）
Video denoising aims at removing noise from videos to recover clean ones. Some existing works show that optical flow can help the denoising by exploiting the additional spatial-temporal clues from nearby frames. However, the flow estimation itself is also sensitive to noise, and can be unusable under large noise levels. To this end, we propose a new multi-scale refined optical flow-guided video denoising method, which is more robust to different noise levels. Our method mainly consists of a denoising-oriented flow refinement (DFR) module and a flow-guided mutual denoising propagation (FMDP) mo…
The vigorous developments of Internet of Things make it possible to extend its computing and storage capabilities to computing tasks in the aerial system with collaboration of cloud and edge, especially for artificial intelligence (AI) tasks based on deep learning (DL). Collecting a large amount of image/video data, Unmanned aerial vehicles (UAVs) can only handover intelligent analysis tasks to the back-end mobile edge computing (MEC) server due to their limited storage and computing capabilities. How to efficiently transmit the most correlated information for the AI model is a challenging top…
University of Chinese Academy of Sciences（中国科学院大学）；CAS Center for Excellence in Brain Science and Intelligence Technology（中国科学院脑科学与智能技术卓越创新中心）
Rotation-invariant face detection, i.e. detecting faces with arbitrary rotation-in-plane (RIP) angles, is widely required in unconstrained applications but still remains as a challenging task, due to the large variations of face appearances. Most existing methods compromise with speed or accuracy to handle the large RIP variations. To address this problem more efficiently, we propose Progressive Calibration Networks (PCN) to perform rotation-invariant face detection in a coarse-to-fine manner. PCN consists of three stages, each of which not only distinguishes the faces from non-faces, but also…
USC: Uncompromising Spatial Constraints for Safety-Oriented 3D Object Detectors in Autonomous Driving
DENSO AUTOMOTIVE Deutschland GmbH（电装汽车德国有限公司）；Technical University of Munich（慕尼黑工业大学）；Chalmers University of Technology（查尔姆斯理工大学）；University of Gothenburg（哥德堡大学）
Dalian University of Technology（大连理工大学）；Beijing Technology And Business University（北京工商大学）；Wonxing Technology（万兴科技）
Most matting researches resort to advanced semantics to achieve high-quality alpha mattes, and direct low-level features combination is usually explored to complement alpha details. However, we argue that appearance-agnostic integration can only provide biased foreground details and alpha mattes require different-level feature aggregation for better pixel-wise opacity perception. In this paper, we propose an end-to-end Hierarchical and Progressive Attention Matting Network (HAttMatting++), which can better predict the opacity of the foreground from single RGB images without additional input. S…
