Although deep learning based image compression methods have achieved promising progress these days, the performance of these methods still cannot match the latest compression standard Versatile Video Coding (VVC). Most of the recent developments focus on designing a more accurate and flexible entropy model that can better parameterize the distributions of the latent features. However, few efforts are devoted to structuring a better transformation between the image space and the latent feature space. In this paper, instead of employing previous autoencoder style networks to build this transformation, we propose an enhanced Invertible Encoding Network with invertible neural networks (INNs) to largely mitigate the information loss problem for better compression. Experimental results on the Kodak, CLIC, and Tecnick datasets show that our method outperforms the existing learned image compression methods and compression standards, including VVC (VTM 12.1), especially for high-resolution images. Our source code is available at https://github.com/xyq7/InvCompress.
Huazhong University of Science and Technology（华中科技大学）
Learning-based methods have effectively promoted the community of image compression. Meanwhile, variational autoencoder (VAE) based variable-rate approaches have recently gained much attention to avoid the usage of a set of different networks for various compression rates. Despite the remarkable performance that has been achieved, these approaches would be readily corrupted once multiple compression/decompression operations are executed, resulting in the fact that image quality would be tremendously dropped and strong artifacts would appear. Thus, we try to tackle the issue of high-fidelity fi…
Bytedance Inc., Shenzhen, China（字节跳动有限公司（深圳））；Bytedance Inc., San Diego, CA, 92122 USA（字节跳动有限公司（圣迭戈））
High-resolution (HR) images are usually downscaled to low-resolution (LR) ones for better display and afterward upscaled back to the original size to recover details. Recent work in image rescaling formulates downscaling and upscaling as a unified task and learns a bijective mapping between HR and LR via invertible networks. However, in real-world applications (e.g., social media), most images are compressed for transmission. Lossy compression will lead to irreversible information loss on LR images, hence damaging the inverse upscaling procedure and degrading the reconstruction accuracy. In th…
Multi-Scale Invertible Neural Network for Wide-Range Variable-Rate Learned Image Compression
National Engineering Laboratory for Brain-Inspired Intelligence Technology and Application, University of Science and Technology of China（中国科学技术大学类脑智能技术及应用国家工程实验室）；University of Missouri（密苏里大学）
Lidars are depth measuring sensors widely used in autonomous driving and augmented reality. However, the large volume of data produced by lidars can lead to high costs in data storage and transmission. While lidar data can be represented as two interchangeable representations: 3D point clouds and range images, most previous work focus on compressing the generic 3D point clouds. In this work, we show that directly compressing the range images can leverage the lidar scanning pattern, compared to compressing the unprojected point clouds. We propose a novel data-driven range image compression algo…
In many professional fields, such as medicine, remote sensing and sciences, users often demand image compression methods to be mathematically lossless. But lossless image coding has a rather low compression ratio (around 2:1 for natural images). The only known technique to achieve significant compression while meeting the stringent fidelity requirements is the methodology of $\ell_\infty$-constrained coding that was developed and standardized in nineties. We make a major progress in $\ell_\infty$-constrained image coding after two decades, by developing a novel CNN-based soft $\ell_\infty$-con…
Peking University（北京大学）；Microsoft Research Asia（微软亚洲研究院）；Peng Cheng Laboratory（鹏城实验室）
Image rescaling is a commonly used bidirectional operation, which first downscales high-resolution images to fit various display screens or to be storage- and bandwidth-friendly, and afterward upscales the corresponding low-resolution images to recover the original resolution or the details in the zoom-in images. However, the non-injective downscaling mapping discards high-frequency contents, leading to the ill-posed problem for the inverse restoration task. This can be abstracted as a general image degradation-restoration problem with information loss. In this work, we propose a novel inverti…
Microsoft Research Asia（微软亚洲研究院）；University of Toronto（多伦多大学）；Peking University（北京大学）
Lossy image compression is one of the most commonly used operators for digital images. Most recently proposed deep-learning-based image compression methods leverage the auto-encoder structure, and reach a series of promising results in this field. The images are encoded into low dimensional latent features first, and entropy coded subsequently by exploiting the statistical redundancy. However, the information lost during encoding is unfortunately inevitable, which poses a significant challenge to the decoder to reconstruct the original images. In this work, we propose a novel invertible framew…
Learning-based image compression has improved to a level where it can outperform traditional image codecs such as HEVC and VVC in terms of coding performance. In addition to good compression performance, device interoperability is essential for a compression codec to be deployed, i.e., encoding and decoding on different CPUs or GPUs should be error-free and with negligible performance reduction. In this paper, we present a method to solve the device interoperability problem of a state-of-the-art image compression network. We implement quantization to entropy networks which output entropy param…
Great successes have been achieved using deep learning techniques for image super-resolution (SR) with fixed scales. To increase its real world applicability, numerous models have also been proposed to restore SR images with arbitrary scale factors, including asymmetric ones where images are resized to different scales along horizontal and vertical directions. Though most models are only optimized for the unidirectional upscaling task while assuming a predefined downscaling kernel for low-resolution (LR) inputs, recent models based on Invertible Neural Networks (INN) are able to increase upsca…
One critical component in lossy deep image compression is the entropy model, which predicts the probability distribution of the quantized latent representation in the encoding and decoding modules. Previous works build entropy models upon convolutional neural networks which are inefficient in capturing global dependencies. In this work, we propose a novel transformer-based entropy model, termed Entroformer, to capture long-range dependencies in probability distribution estimation effectively and efficiently. Different from vision transformers in image classification, the Entroformer is highly …
Recent deep learning models outperform standard lossy image compression codecs. However, applying these models on a patch-by-patch basis requires that each image patch be encoded and decoded independently. The influence from adjacent patches is therefore lost, leading to block artefacts at low bitrates. We propose the Binary Inpainting Network (BINet), an autoencoder framework which incorporates binary inpainting to reinstate interdependencies between adjacent patches, for improved patch-based compression of still images. When decoding a patch, BINet additionally uses the binarised encodings f…
In this paper a new fractal image compression algorithm is proposed in which the time of encoding process is considerably reduced. The algorithm exploits a domain pool reduction approach, along with using innovative predefined values for contrast scaling factor, S, instead of scanning the parameter space [0,1]. Within this approach only domain blocks with entropies greater than a threshold are considered. As a novel point, it is assumed that in each step of the encoding process, the domain block with small enough distance shall be found only for the range blocks with low activity (equivalently…
Fractal image compression is attractive except for its high encoding time requirements. The image is encoded as a set of contractive affine transformations. The image is partitioned into non-overlapping range blocks, and a best matching domain block larger than the range block is identified. There are many attempts on improving the encoding time by reducing the size of search pool for range-domain matching. But these methods are attempting to prepare a static domain pool that remains unchanged throughout the encoding process. This paper proposes dynamic preparation of separate domain pool for …
In this paper a new fractal image compression algorithm is proposed in which the time of encoding process is considerably reduced. The algorithm exploits a domain pool reduction approach, along with using innovative predefined values for contrast scaling factor, S, instead of searching it across [0,1]. Only the domain blocks with entropy greater than a threshold are considered as domain pool. As a novel point, it is assumed that in each step of the encoding process, the domain block with small enough distance shall be found only for the range blocks with low activity (equivalently low entropy)…
ETH Zurich（苏黎世联邦理工学院）；Technical University of Munich（慕尼黑工业大学）；Google（谷歌）
Recently Implicit Neural Representations (INRs) gained attention as a novel and effective representation for various data types. Thus far, prior work mostly focused on optimizing their reconstruction performance. This work investigates INRs from a novel perspective, i.e., as a tool for image compression. To this end, we propose the first comprehensive compression pipeline based on INRs including quantization, quantization-aware retraining and entropy coding. Encoding with INRs, i.e. overfitting to a data sample, is typically orders of magnitude slower. To mitigate this drawback, we leverage me…
Compression as an Adversarial Amplifier Through Decision Space Reduction
University of Nottingham（诺丁汉大学）；University of Chinese Academy of Sciences（中国科学院大学）；Xiamen University（厦门大学）；School of Computer Science, University of Nottingham（诺丁汉大学计算机科学学院）
Fractal image compression has some desirable properties like high quality at high compression ratio, fast decoding, and resolution independence. Therefore it can be used for many applications such as texture mapping and pattern recognition and image watermarking. But it suffers from long encoding time due to its need to find the best match between sub blocks. This time is related to the approach that is used. In this paper we present a fast encoding Algorithm based on no search method. Our goal is that more blocks are covered in initial step of quad tree algorithm. Experimental result has been…
Matoshri College of Engineering and Research Centre（马托什里工程与研究中心学院）；University of Pune（浦那大学）
In Image Compression, the researchers' aim is to reduce the number of bits required to represent an image by removing the spatial and spectral redundancies. Recently discrete wavelet transform and wavelet packet has emerged as popular techniques for image compression. The wavelet transform is one of the major processing components of image compression. The result of the compression changes as per the basis and tap of the wavelet used. It is proposed that proper selection of mother wavelet on the basis of nature of images, improve the quality as well as compression ratio remarkably. We suggest …
University of Naples "Parthenope"（那不勒斯帕尔泰诺佩大学）；University of Salerno（萨勒诺大学）
The compression is an important topic in computer science which allows we to storage more amount of data on our data storage. There are several techniques to compress any file. In this manuscript will be described the most important algorithm to compress images such as JPEG and it will be compared with another method to retrieve good reason to not use this method on images. So to compress the text the most encoding technique known is the Huffman Encoding which it will be explained in exhaustive way. In this manuscript will showed how to compute a text compression method on images in particular…
TVRN: Invertible Neural Networks for Compression-Aware Temporal Video Rescaling
MOE Key Laboratory of Brain-Inspired Intelligent Perception and Cognition（教育部脑科学与智能感知认知重点实验室）；University of Science and Technology of China（中国科学技术大学）；Information Science and Technology Institution（信息科学与技术研究所）；MCC Lab（MCC实验室）
Indian Institute of Technology Madras（印度理工学院马德拉斯分校）
The emerging and existing light field displays are highly capable of realistic presentation of 3D scenes on auto-stereoscopic glasses-free platforms. The light field size is a major drawback while utilising 3D displays and streaming purposes. When a light field is of high dynamic range, the size increases drastically. In this paper, we propose a novel compression algorithm for a high dynamic range light field which yields a perceptually lossless compression. The algorithm exploits the inter and intra view correlations of the HDR light field by interpreting it to be a four-dimension volume. The…
Zhejiang University（浙江大学）；University of Electronic Science and Technology of China（电子科技大学）
The data storage has been one of the bottlenecks in surveillance systems. The conventional video compression algorithms such as H.264 and H.265 do not fully utilize the low information density characteristic of the surveillance video. In this paper, we propose a video compression method that extracts and compresses the foreground and background of the video separately. The compression ratio is greatly improved by sharing background information among multiple adjacent frames through an adaptive background updating and interpolation module. Besides, we present two different schemes to compress t…
We propose a new paradigm for encoding the geometry of point cloud sequences, where the convolutional neural network (CNN) which estimates the encoding distributions is optimized on several frames of the sequence to be compressed. We adopt lightweight CNN structures, we perform training as part of the encoding process, and the CNN parameters are transmitted as part of the bitstream. The newly proposed encoding scheme operates on the octree representation for each point cloud, encoding consecutively each octree resolution layer. At every octree resolution layer, the voxel grid is traversed sect…
The Hong Kong University of Science and Technology（香港科技大学）
Unprocessed RAW data is a highly valuable image format for image editing and computer vision. However, since the file size of RAW data is huge, most users can only get access to processed and compressed sRGB images. To bridge this gap, we design an Invertible Image Signal Processing (InvISP) pipeline, which not only enables rendering visually appealing sRGB images but also allows recovering nearly perfect RAW data. Due to our framework's inherent reversibility, we can reconstruct realistic RAW data instead of synthesizing RAW data from sRGB images without any memory overhead. We also integrate…
The Chinese University of Hong Kong（香港中文大学）；Caritas Institute of Higher Education（明爱专上学院）
Although digital cameras can acquire high-dynamic range (HDR) images, the captured HDR information are mostly quantized to low-dynamic range (LDR) images for display compatibility and compact storage. In this paper, we propose an invertible tone mapping method that converts the multi-exposure HDR to a true LDR (8-bit per color channel) and reserves the capability to accurately restore the original HDR from this {\em invertible LDR}. Our invertible LDR can mimic the appearance of a user-selected tone mapping style. It can be shared over any existing social network platforms that may re-encode o…
The dissertation proposes the use of a multi-objective optimization framework for designing and selecting among enhanced GOP configurations in video compression standards. The proposed methods achieve fine optimization over a set of general modes that include: (i) maximum video quality, (ii) minimum bitrate, (iii) maximum encoding rate (previously minimum encoding time mode) and (iv) can be shown to improve upon the YouTube/Netflix default encoder mode settings over a set of opposing constraints to guarantee satisfactory performance. The dissertation describes the implementation of a codec-agn…
Leveraging Compression to Construct Transferable Bitrate Ladders
Laboratory for Image and Video Engineering (LIVE), The University of Texas at Austin（图像与视频工程实验室（LIVE），德克萨斯大学奥斯汀分校）；Colorado’s Laboratory for Image and Video Engineering (LIVE), University of Colorado Boulder（科罗拉多州图像与视频工程实验室（LIVE），科罗拉多大学波德分校）；Meta Platforms Inc.（Meta平台公司）
Encoding textural content remains a challenge for current standardised video codecs. It is therefore beneficial to understand video textures in terms of both their spatio-temporal characteristics and their encoding statistics in order to optimize encoding performance. In this paper, we analyse the spatio-temporal features and statistics of video textures, explore the rate-quality performance of different texture types and investigate models to mathematically describe them. For all considered theoretical models, we employ machine-learning regression to predict the rate-quality curves based sole…
Three-dimensional (3D) shape measurement devices and techniques are being rapidly adopted within a variety of industries and applications. As acquiring 3D range data becomes faster and more accurate it becomes more challenging to efficiently store, transmit, or stream this data. One prevailing approach to compressing 3D range data is to encode it within the color channels of regular 2D images. This paper presents a novel method for reducing the depth range of a 3D geometry such that it can be stored within a 2D image using lower encoding frequencies (or a fewer number of encoding periods). Thi…
Carnegie Mellon University（卡内基梅隆大学）；University of Southern California（南加州大学）；University of California, Berkeley（加利福尼亚大学伯克利分校）
In this work, we propose a new generative model that is capable of automatically decoupling global and local representations of images in an entirely unsupervised setting, by embedding a generative flow in the VAE framework to model the decoder. Specifically, the proposed model utilizes the variational auto-encoding framework to learn a (low-dimensional) vector of latent variables to capture the global information of an image, which is fed as a conditional input to a flow-based invertible decoder with architecture borrowed from style transfer literature. Experimental results on standard image …
Variational Autoencoders (VAEs) have seen widespread use in learned image compression. They are used to learn expressive latent representations on which downstream compression methods can operate with high efficiency. Recently proposed 'bits-back' methods can indirectly encode the latent representation of images with codelength close to the relative entropy between the latent posterior and the prior. However, due to the underlying algorithm, these methods can only be used for lossless compression, and they only achieve their nominal efficiency when compressing multiple images simultaneously; t…
