Over the past several years, we have witnessed impressive progress in the field of learned image compression. Recent learned image codecs are commonly based on autoencoders, that first encode an image into low-dimensional latent representations and then decode them for reconstruction purposes. To capture spatial dependencies in the latent space, prior works exploit hyperprior and spatial context model to build an entropy model, which estimates the bit-rate for end-to-end rate-distortion optimization. However, such an entropy model is suboptimal from two aspects: (1) It fails to capture spatially global correlations among the latents. (2) Cross-channel relationships of the latents are still underexplored. In this paper, we propose the concept of separate entropy coding to leverage a serial decoding process for causal contextual entropy prediction in the latent space. A causal context model is proposed that separates the latents across channels and makes use of cross-channel relationships to generate highly informative contexts. Furthermore, we propose a causal global prediction model, which is able to find global reference points for accurate predictions of unknown points. Both these two models facilitate entropy estimation without the transmission of overhead. In addition, we further adopt a new separate attention module to build more powerful transform networks. Experimental results demonstrate that our full image compression model outperforms standard VVC/H.266 codec on Kodak dataset in terms of both PSNR and MS-SSIM, yielding the state-of-the-art rate-distortion performance.
Recently, learned image compression techniques have achieved remarkable performance, even surpassing the best manually designed lossy image coders. They are promising to be large-scale adopted. For the sake of practicality, a thorough investigation of the architecture design of learned image compression, regarding both compression performance and running speed, is essential. In this paper, we first propose uneven channel-conditional adaptive coding, motivated by the observation of energy compaction in learned image compression. Combining the proposed uneven grouping model with existing context…
UCVC: A Unified Contextual Video Compression Framework with Joint P-frame and B-frame Coding
School of Electronic and Computer Engineering, Peking University（北京大学电子与计算机工程学院）；Peng Cheng Laboratory（鹏城实验室）；Migu Culture Technology Co., Ltd（咪咕文化科技有限公司）
Fractal image compression, Culik's image compression and zerotree prediction coding of wavelet image decomposition coefficients succeed only because typical images being compressed possess a significant degree of self-similarity. Besides the common concept, these methods turn out to be even more tightly related, to the point of algorithmical reducibility of one technique to another. The goal of the present paper is to demonstrate these relations. The paper offers a plain-term interpretation of Culik's image compression, in regular image processing terms, without resorting to finite state machi…
HSRMamba: Contextual Spatial-Spectral State Space Model for Single Image Hyperspectral Super-Resolution
National Engineering Research Center for Multimedia Software, School of Computer Science, Wuhan University（国家多媒体软件工程研究中心，计算机学院，武汉大学）；Aerospace Information Research Institute, Henan Academy of Sciences（航天信息研究所，河南科学院）
GroupedMixer: An Entropy Model with Group-wise Token-Mixers for Learned Image Compression
School of Computer Science and Technology, Harbin Institute of Technology（哈尔滨工业大学计算机科学与技术学院）；School of Electronics Engineering and Computer Science, Peking University（北京大学电子工程与计算机科学学院）
Rochester Institute of Technology（罗切斯特理工学院）；Pennsylvania State University（宾夕法尼亚州立大学）；Old Dominion University（奥尔德多米宁大学）
For lossy image compression systems, we develop an algorithm, iterative refinement, to improve the decoder's reconstruction compared to standard decoding techniques. Specifically, we propose a recurrent neural network approach for nonlinear, iterative decoding. Our decoder, which works with any encoder, employs self-connected memory units that make use of causal and non-causal spatial context information to progressively reduce reconstruction error over a fixed number of steps. We experiment with variants of our estimator and find that iterative refinement consistently creates lower distortion…
Most of the existing neural video compression methods adopt the predictive coding framework, which first generates the predicted frame and then encodes its residue with the current frame. However, as for compression ratio, predictive coding is only a sub-optimal solution as it uses simple subtraction operation to remove the redundancy across frames. In this paper, we propose a deep contextual video compression framework to enable a paradigm shift from predictive coding to conditional coding. In particular, we try to answer the following questions: how to define, use, and learn condition under …
Causality knowledge is crucial for many artificial intelligence systems. Conventional textual-based causality knowledge acquisition methods typically require laborious and expensive human annotations. As a result, their scale is often limited. Moreover, as no context is provided during the annotation, the resulting causality knowledge records (e.g., ConceptNet) typically do not take the context into consideration. To explore a more scalable way of acquiring causality knowledge, in this paper, we jump out of the textual domain and investigate the possibility of learning contextual causality fro…
DCVC-MV: Deep Contextual Multiview Video Compression with Efficient Inter-View Prediction
Department of Computer Science, City University of Hong Kong（城市大学计算机科学系）；Faculty of Electrical Engineering and Computer Science, Ningbo University（宁波大学电子工程与计算机科学学院）
University of Science and Technology of China（中国科学技术大学）；Institute of Artificial Intelligence, Hefei Comprehensive National Science Center（合肥综合性国家科学中心人工智能研究院）；City University of Hong Kong（香港城市大学）
PACE: Post-Causal Entropy Modeling for Learned LiDAR Point Cloud Compression
School of Information Science and Technology, Hangzhou Normal University, Hangzhou, China.（信息科学与技术学院，杭州师范大学，杭州，中国）；School of Electronic Science and Engineering, Nanjing University, Nanjing, China（电子科学与工程学院，南京大学，南京，中国）
Deep learning solutions of the salient object detection problem have achieved great results in recent years. The majority of these models are based on encoders and decoders, with a different multi-feature combination. In this paper, we show that feature concatenation works better than other combination methods like multiplication or addition. Also, joint feature learning gives better results, because of the information sharing during their processing. We designed a Complementary Extraction Module (CEM) to extract necessary features with edge preservation. Our proposed Excessiveness Loss (EL) f…
Digital Elevation models (DEM) are images having terrain information embedded into them. Using cognitive mapping concepts for DEM registration, has evolved from this basic idea of using the mapping between the space to objects and defining their relationships to form the basic landmarks that need to be marked, stored and manipulated in and about the environment or other candidate environments, namely, in our case, the DEMs. The progressive two-level encapsulation of methods of geo-spatial cognition includes landmark knowledge and layout knowledge and can be useful for DEM registration. Space-b…
National Laboratory of Pattern Recognition（模式识别国家重点实验室）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）
Human face synthesis involves transferring knowledge about the identity and identity-dependent face shape (IDFS) of a human face to target face images where the context (e.g., facial expressions, head poses, and other background factors) may change dramatically. Human faces are non-rigid, so facial expression leads to deformation of face shape, and head pose also affects the face observed in 2D images. A key challenge in face transfer is to match the face with unobserved new contexts, adapting the face appearance to different poses and expressions accordingly. In this work, we find a way to pr…
Academy for Engineering and Technology, Fudan University（复旦大学工程与应用技术研究院）；Institute of Meta-Medical, IPASS（IPASS元医学研究所）；Jilin Provincial Key Laboratory of Intelligence Science and Engineering（吉林省智能科学与工程重点实验室）；Engineering Research Center of AI and Robotics, Ministry of Education（教育部人工智能与机器人工程研究中心）
Context-Aware Emotion Recognition (CAER) is a crucial and challenging task that aims to perceive the emotional states of the target person with contextual information. Recent approaches invariably focus on designing sophisticated architectures or mechanisms to extract seemingly meaningful representations from subjects and contexts. However, a long-overlooked issue is that a context bias in existing datasets leads to a significantly unbalanced distribution of emotional states among different context scenarios. Concretely, the harmful bias is a confounder that misleads existing models to learn s…
