In this paper, we propose a novel convolutional neural network (CNN) architecture, MFRNet, for post-processing (PP) and in-loop filtering (ILF) in the context of video compression. This network consists of four Multi-level Feature review Residual dense Blocks (MFRBs), which are connected using a cascading structure. Each MFRB extracts features from multiple convolutional layers using dense connections and a multi-level residual learning structure. In order to further improve information flow between these blocks, each of them also reuses high dimensional features from the previous MFRB. This network has been integrated into PP and ILF coding modules for both HEVC (HM 16.20) and VVC (VTM 7.0), and fully evaluated under the JVET Common Test Conditions using the Random Access configuration. The experimental results show significant and consistent coding gains over both anchor codecs (HEVC HM and VVC VTM) and also over other existing CNN-based PP/ILF approaches based on Bjontegaard Delta measurements using both PSNR and VMAF for quality assessment. When MFRNet is integrated into HM 16.20, gains up to 16.0% (BD-rate VMAF) are demonstrated for ILF, and up to 21.0% (BD-rate VMAF) for PP. The respective gains for VTM 7.0 are up to 5.1% for ILF and up to 7.1% for PP.
In-loop filtering is used in video coding to process the reconstructed frame in order to remove blocking artifacts. With the development of convolutional neural networks (CNNs), CNNs have been explored for in-loop filtering considering it can be treated as an image de-noising task. However, in addition to being a distorted image, the reconstructed frame is also obtained by a fixed line of block based encoding operations in video coding. It carries coding-unit based coding distortion of some similar characteristics. Therefore, in this paper, we address the filtering problem from two aspects, gl…
With the advances of fifth-generation (5G) cellular networks technology, many studies and work have been carried out on how to build 5G networks for smart cities. In the previous research, street lighting poles and smart light poles are capable of being a 5G access point. In order to determine the position of the points, this paper discusses a new way to identify poles based on Mask R-CNN, which extends Fast R-CNNs by making it employ recursive Bayesian filtering and perform proposal propagation and reuse. The dataset contains 3,000 high-resolution images from google map. To make training fast…
The lossy compression techniques produce various artifacts like blurring, distortion at block bounders, ringing and contouring effects on outputs especially at low bit rates. To reduce those compression artifacts various Convolutional Neural Network (CNN) based post processing techniques have been experimented over recent years. The latest video coding standard HEVC adopts two post processing filtering operations namely de-blocking filter (DBF) followed by sample adaptive offset (SAO). These operations consumes extra signaling bit and becomes an overhead to network. In this paper we proposed a…
Face recognition is one of the most active tasks in computer vision and has been widely used in the real world. With great advances made in convolutional neural networks (CNN), lots of face recognition algorithms have achieved high accuracy on various face datasets. However, existing face recognition algorithms based on CNNs are vulnerable to noise. Noise corrupted image patterns could lead to false activations, significantly decreasing face recognition accuracy in noisy situations. To equip CNNs with built-in robustness to noise of different levels, we proposed a Median Pixel Difference Convo…
York University（约克大学）；Weizmann Institute of Science（魏茨曼科学研究所）
While deep learning (DL) architectures like convolutional neural networks (CNNs) have enabled effective solutions in image denoising, in general their implementations overly rely on training data, lack interpretability, and require tuning of a large parameter set. In this paper, we combine classical graph signal filtering with deep feature learning into a competitive hybrid design -- one that utilizes interpretable analytical low-pass graph filters and employs 80% fewer network parameters than state-of-the-art DL denoising scheme DnCNN. Specifically, to construct a suitable similarity graph fo…
Friedrich-Alexander University of Erlangen-Nuremberg（埃尔朗根-纽伦堡大学）；Siemens Healthcare GmbH（西门子医疗有限公司）
Denoising of clinical CT images is an active area for deep learning research. Current clinically approved methods use iterative reconstruction methods to reduce the noise in CT images. Iterative reconstruction techniques require multiple forward and backward projections, which are time-consuming and computationally expensive. Recently, deep learning methods have been successfully used to denoise CT images. However, conventional deep learning methods suffer from the 'black box' problem. They have low accountability, which is necessary for use in clinical imaging situations. In this paper, we us…
Sun Yat-sen University（中山大学）；South China University of Technology（华南理工大学）；Hunan University（湖南大学）；University of Extremadura（埃斯特雷马杜拉大学）；Beihang University（北京航空航天大学）
Recently, many convolutional neural network (CNN) methods have been designed for hyperspectral image (HSI) classification since CNNs are able to produce good representations of data, which greatly benefits from a huge number of parameters. However, solving such a high-dimensional optimization problem often requires a large amount of training samples in order to avoid overfitting. Additionally, it is a typical non-convex problem affected by many local minima and flat regions. To address these problems, in this paper, we introduce naive Gabor Networks or Gabor-Nets which, for the first time in t…
Convolutional Neural Network (CNN)-based filters have achieved significant performance in video artifacts reduction. However, the high complexity of existing methods makes it difficult to be applied in real usage. In this paper, a CNN-based low complexity filter is proposed. We utilize depth separable convolution (DSC) merged with the batch normalization (BN) as the backbone of our proposed CNN-based network. Besides, a weight initialization method is proposed to enhance the training performance. To solve the well known over smoothing problem for the inter frames, a frame-level residual mappin…
Syed Babar Ali School of Science and Engineering (SBASSE), Lahore University of Management Sciences (LUMS)（拉合尔管理科学大学赛义德·巴巴尔·阿里理工学院（SBASSE））；Department of Electrical Engineering, Syed Babar Ali School of Science and Engineering (SBASSE), Lahore University of Management Sciences (LUMS)（拉合尔管理科学大学赛义德·巴巴尔·阿里理工学院电气工程系（SBASSE））；Lahore University of Management Sciences (LUMS)（拉合尔管理科学大学（LUMS））；Department of Electrical Engineering（电气工程系）
Convolutional Neural Networks (CNNs) have gained widespread popularity in the field of computer vision and image processing. Due to huge computational requirements of CNNs, dedicated hardware-based implementations are being explored to improve their performance. Hardware platforms such as Field Programmable Gate Arrays (FPGAs) are widely being used to design parallel architectures for this purpose. In this paper, we analyze Winograd minimal filtering or fast convolution algorithms to reduce the arithmetic complexity of convolutional layers of CNNs. We explore a complex design space to find the…
Deep learning methods are increasingly being applied in the optimisation of video compression algorithms and can achieve significantly enhanced coding gains, compared to conventional approaches. Such approaches often employ Convolutional Neural Networks (CNNs) which are trained on databases with relatively limited content coverage. In this paper, a new extensive and representative video database, BVI-DVC, is presented for training CNN-based video compression systems, with specific emphasis on machine learning tools that enhance conventional coding architectures, including spatial resolution an…
Artifact removal and filtering methods are inevitable parts of video coding. On one hand, new codecs and compression standards come with advanced in-loop filters and on the other hand, displays are equipped with high capacity processing units for post-treatment of decoded videos. This paper proposes a Convolutional Neural Network (CNN)-based post-processing algorithm for intra and inter frames of Versatile Video Coding (VVC) coded streams. Depending on the frame type, this method benefits from normative prediction signal by feeding it as an additional input along with reconstructed signal and …
This paper presents a framework for Convolutional Neural Network (CNN)-based quality enhancement task, by taking advantage of coding information in the compressed video signal. The motivation is that normative decisions made by the encoder can significantly impact the type and strength of artifacts in the decoded images. In this paper, the main focus has been put on decisions defining the prediction signal in intra and inter frames. This information has been used in the training phase as well as input to help the process of learning artifacts that are specific to each coding type. Furthermore,…
Technical University of Munich (TUM)（慕尼黑工业大学（TUM））；German Aerospace Center (DLR)（德国航空航天中心（DLR））
City models and height maps of urban areas serve as a valuable data source for numerous applications, such as disaster management or city planning. While this information is not globally available, it can be substituted by digital surface models (DSMs), automatically produced from inexpensive satellite imagery. However, stereo DSMs often suffer from noise and blur. Furthermore, they are heavily distorted by vegetation, which is of lesser relevance for most applications. Such basic models can be filtered by convolutional neural networks (CNNs), trained on labels derived from digital elevation m…
Deakin University（迪肯大学）；University of Birjand（比尔詹德大学）；Ferdowsi University of Mashhad（费尔多西大学）；K. N. Toosi University of Technology（KN图西理工大学）；Semnan University（塞姆南大学）；Dibrugarh University（迪布鲁格尔大学）；Iran University of Medical Sciences（伊朗医科大学）；University of California, San Francisco（加州大学旧金山分校）；The George Institute for Global Health（乔治全球健康研究院）；University of Sydney（悉尼大学）；Ngee Ann Polytechnic（义安理工学院）；Singapore University of Social Sciences（新加坡新跃社科大学）；Asia University（亚洲大学）
The coronavirus (COVID-19) is currently the most common contagious disease which is prevalent all over the world. The main challenge of this disease is the primary diagnosis to prevent secondary infections and its spread from one person to another. Therefore, it is essential to use an automatic diagnosis system along with clinical procedures for the rapid diagnosis of COVID-19 to prevent its spread. Artificial intelligence techniques using computed tomography (CT) images of the lungs and chest radiography have the potential to obtain high diagnostic performance for Covid-19 diagnosis. In this …
