Sparse Multi-layer Image Approximation: Facial Image Compression

A Boosting Method to Face Image Super-resolution

Recently sparse representation has gained great success in face image super-resolution. The conventional sparsity-based methods enforce sparse coding on face image patches and the representation fidelity is measured by $\ell_{2}$-norm. Such a sparse coding model regularizes all facial patches equally, which however ignores distinct natures of different facial patches for image reconstruction. In this paper, we propose a new weighted-patch super-resolution method based on AdaBoost. Specifically, in each iteration of the AdaBoost operation, each facial patch is weighted automatically according t…

FaSDiff：通过Stable Diffusion先验平衡人脸压缩中的感知与语义

FaSDiff: Balancing Perception and Semantics in Face Compression via Stable Diffusion Priors

针对低比特率下人脸压缩重建质量低、扩散先验应用导致下游任务性能差的问题，提出FaSDiff框架，通过高频敏感压缩器和混合低频增强模块平衡视觉保真度与语义一致性，实验证明其优于现有方法。

A Framework for Super-Resolution of Scalable Video via Sparse Reconstruction of Residual Frames

This paper introduces a framework for super-resolution of scalable video based on compressive sensing and sparse representation of residual frames in reconnaissance and surveillance applications. We exploit efficient compressive sampling and sparse reconstruction algorithms to super-resolve the video sequence with respect to different compression rates. We use the sparsity of residual information in residual frames as the key point in devising our framework. Moreover, a controlling factor as the compressibility threshold to control the complexity-performance trade-off is defined. Numerical exp…

使用掩码稀疏视觉表示的神经图像压缩

Neural Image Compression Using Masked Sparse Visual Representation

针对基于SVR的图像压缩难以实现率失真权衡的问题，本文提出M-AdaCode，通过语义类别基码本自适应加权及掩码与恢复机制控制比特率和失真，并在JPEG-AI上验证有效性。

Regularized Residual Quantization: a multi-layer sparse dictionary learning approach

The Residual Quantization (RQ) framework is revisited where the quantization distortion is being successively reduced in multi-layers. Inspired by the reverse-water-filling paradigm in rate-distortion theory, an efficient regularization on the variances of the codewords is introduced which allows to extend the RQ for very large numbers of layers and also for high dimensional data, without getting over-trained. The proposed Regularized Residual Quantization (RRQ) results in multi-layer dictionaries which are additionally sparse, thanks to the soft-thresholding nature of the regularization when …

Metric Learning for Anti-Compression Facial Forgery Detection

Detecting facial forgery images and videos is an increasingly important topic in multimedia forensics. As forgery images and videos are usually compressed into different formats such as JPEG and H264 when circulating on the Internet, existing forgery-detection methods trained on uncompressed data often suffer from significant performance degradation in identifying them. To solve this problem, we propose a novel anti-compression facial forgery detection framework, which learns a compression-insensitive embedding feature space utilizing both original and compressed forgeries. Specifically, our a…

End-to-End Facial Deep Learning Feature Compression with Teacher-Student Enhancement

In this paper, we propose a novel end-to-end feature compression scheme by leveraging the representation and learning capability of deep neural networks, towards intelligent front-end equipped analysis with promising accuracy and efficiency. In particular, the extracted features are compactly coded in an end-to-end manner by optimizing the rate-distortion cost to achieve feature-in-feature representation. In order to further improve the compression performance, we present a latent code level teacher-student enhancement model, which could efficiently transfer the low bit-rate representation int…

The Effect of Model Compression on Fairness in Facial Expression Recognition

Deep neural networks have proved hugely successful, achieving human-like performance on a variety of tasks. However, they are also computationally expensive, which has motivated the development of model compression techniques which reduce the resource consumption associated with deep learning models. Nevertheless, recent studies have suggested that model compression can have an adverse effect on algorithmic fairness, amplifying existing biases in machine learning models. With this project we aim to extend those studies to the context of facial expression recognition. To do that, we set up a ne…

面向面部表情识别的优化学习图像压缩

Optimized Learned Image Compression for Facial Expression Recognition

本文提出一种面向面部表情识别的端到端学习图像压缩模型，通过自定义损失函数联合优化压缩与识别性能，实验显示准确率和压缩效率分别提升4.04%和89.12%。

Spontaneous Facial Expression Recognition using Sparse Representation

Facial expression is the most natural means for human beings to communicate their emotions. Most facial expression analysis studies consider the case of acted expressions. Spontaneous facial expression recognition is significantly more challenging since each person has a different way to react to a given emotion. We consider the problem of recognizing spontaneous facial expression by learning discriminative dictionaries for sparse representation. Facial images are represented as a sparse linear combination of prototype atoms via Orthogonal Matching Pursuit algorithm. Sparse codes are then used…

Anti-Compression Contrastive Facial Forgery Detection

Forgery facial images and videos have increased the concern of digital security. It leads to the significant development of detecting forgery data recently. However, the data, especially the videos published on the Internet, are usually compressed with lossy compression algorithms such as H.264. The compressed data could significantly degrade the performance of recent detection algorithms. The existing anti-compression algorithms focus on enhancing the performance in detecting heavily compressed data but less consider the compression adaption to the data from various compression levels. We bel…

Facial Expression Recognition Using Sparse Gaussian Conditional Random Field

The analysis of expression and facial Action Units (AUs) detection are very important tasks in fields of computer vision and Human Computer Interaction (HCI) due to the wide range of applications in human life. Many works has been done during the past few years which has their own advantages and disadvantages. In this work we present a new model based on Gaussian Conditional Random Field. We solve our objective problem using ADMM and we show how well the proposed model works. We train and test our work on two facial expression datasets, CK+ and RU-FACS. Experimental evaluation shows that our p…

当少即是多：一种用于倾听动作学习的稀疏面部运动结构

When Less Is More: A Sparse Facial Motion Structure For Listening Motion Learning

本文提出一种稀疏关键帧与过渡帧编码方法来表示和预测非言语面部运动，以解决离散令牌表示训练低效和生成保真度低的问题，并应用于倾听头部预测任务。

Sparsely Grouped Multi-task Generative Adversarial Networks for Facial Attribute Manipulation

Recent Image-to-Image Translation algorithms have achieved significant progress in neural style transfer and image attribute manipulation tasks. However, existing approaches require exhaustively labelling training data, which is labor demanding, difficult to scale up, and hard to migrate into new domains. To overcome such a key limitation, we propose Sparsely Grouped Generative Adversarial Networks (SG-GAN) as a novel approach that can translate images on sparsely grouped datasets where only a few samples for training are labelled. Using a novel one-input multi-output architecture, SG-GAN is w…

SFDM：面向稀疏视图图像真实感人脸渲染的几何与反射率鲁棒分解

SFDM: Robust Decomposition of Geometry and Reflectance for Realistic Face Rendering from Sparse-view Images

提出两阶段SFDM，先构建通用面部模板，再结合几何-反射率交互与次表面散射细化个体模型，实现三张稀疏视图图像下的高质量人脸分解与真实感渲染。

Local Geometric Indexing of High Resolution Data for Facial Reconstruction from Sparse Markers

When considering sparse motion capture marker data, one typically struggles to balance its overfitting via a high dimensional blendshape system versus underfitting caused by smoothness constraints. With the current trend towards using more and more data, our aim is not to fit the motion capture markers with a parameterized (blendshape) model or to smoothly interpolate a surface through the marker positions, but rather to find an instance in the high resolution dataset that contains local geometry to fit each marker. Just as is true for typical machine learning applications, this approach benef…

A multi-layer network based on Sparse Ternary Codes for universal vector compression

We present the multi-layer extension of the Sparse Ternary Codes (STC) for fast similarity search where we focus on the reconstruction of the database vectors from the ternary codes. To consider the trade-offs between the compactness of the STC and the quality of the reconstructed vectors, we study the rate-distortion behavior of these codes under different setups. We show that a single-layer code cannot achieve satisfactory results at high rates. Therefore, we extend the concept of STC to multiple layers and design the ML-STC, a codebook-free system that successively refines the reconstructio…

Sparse Coding of Shape Trajectories for Facial Expression and Action Recognition

The detection and tracking of human landmarks in video streams has gained in reliability partly due to the availability of affordable RGB-D sensors. The analysis of such time-varying geometric data is playing an important role in the automatic human behavior understanding. However, suitable shape representations as well as their temporal evolution, termed trajectories, often lie to nonlinear manifolds. This puts an additional constraint (i.e., nonlinearity) in using conventional Machine Learning techniques. As a solution, this paper accommodates the well-known Sparse Coding and Dictionary Lear…

面向情感的稀疏注意力：基于令牌缩减的高效面部表情识别

Sparse Attention to Emotion: Efficient Facial Emotion Recognition via Token Reduction

针对现有视觉Transformer面部表情识别方法复杂度高难以边缘部署的问题，提出SAE模型，通过丢弃90%无价值图像令牌实现轻量FER，在RAF-DB数据集上达最优结果且复杂度降90%。

Sparse to Dense Dynamic 3D Facial Expression Generation

In this paper, we propose a solution to the task of generating dynamic 3D facial expressions from a neutral 3D face and an expression label. This involves solving two sub-problems: (i)modeling the temporal dynamics of expressions, and (ii) deforming the neutral mesh to obtain the expressive counterpart. We represent the temporal evolution of expressions using the motion of a sparse set of 3D landmarks that we learn to generate by training a manifold-valued GAN (Motion3DGAN). To better encode the expression-induced deformation and disentangle it from the identity information, the generated moti…

GGViT:Multistream Vision Transformer Network in Face2Face Facial Reenactment Detection

Detecting manipulated facial images and videos on social networks has been an urgent problem to be solved. The compression of videos on social media has destroyed some pixel details that could be used to detect forgeries. Hence, it is crucial to detect manipulated faces in videos of different quality. We propose a new multi-stream network architecture named GGViT, which utilizes global information to improve the generalization of the model. The embedding of the whole face extracted by ViT will guide each stream network. Through a large number of experiments, we have proved that our proposed mo…

FaceForensics++: Learning to Detect Manipulated Facial Images

The rapid progress in synthetic image generation and manipulation has now come to a point where it raises significant concerns for the implications towards society. At best, this leads to a loss of trust in digital content, but could potentially cause further harm by spreading false information or fake news. This paper examines the realism of state-of-the-art image manipulations, and how difficult it is to detect them, either automatically or by humans. To standardize the evaluation of detection methods, we propose an automated benchmark for facial manipulation detection. In particular, the be…

Demography-based Facial Retouching Detection using Subclass Supervised Sparse Autoencoder

Digital retouching of face images is becoming more widespread due to the introduction of software packages that automate the task. Several researchers have introduced algorithms to detect whether a face image is original or retouched. However, previous work on this topic has not considered whether or how accuracy of retouching detection varies with the demography of face images. In this paper, we introduce a new Multi-Demographic Retouched Faces (MDRF) dataset, which contains images belonging to two genders, male and female, and three ethnicities, Indian, Chinese, and Caucasian. Further, retou…

基于双重稀疏光流分解的拉格朗日运动放大

Lagrangian Motion Magnification with Double Sparse Optical Flow Decomposition

本文提出一种面部微运动局部拉格朗日放大方法，通过RAFT光流微调、时空双重稀疏分解及三角剖分前向变形策略，实现高效准确的微表情运动放大。

TAFIM: Targeted Adversarial Attacks against Facial Image Manipulations

Face manipulation methods can be misused to affect an individual's privacy or to spread disinformation. To this end, we introduce a novel data-driven approach that produces image-specific perturbations which are embedded in the original images. The key idea is that these protected images prevent face manipulation by causing the manipulation model to produce a predefined manipulation target (uniformly colored output image in our case) instead of the actual manipulation. In addition, we propose to leverage differentiable compression approximation, hence making generated perturbations robust to c…

Survey on Sparse Coded Features for Content Based Face Image Retrieval

Content based image retrieval, a technique which uses visual contents of image to search images from large scale image databases according to users' interests. This paper provides a comprehensive survey on recent technology used in the area of content based face image retrieval. Nowadays digital devices and photo sharing sites are getting more popularity, large human face photos are available in database. Multiple types of facial features are used to represent discriminality on large scale human facial image database. Searching and mining of facial images are challenging problems and important…

基于wing损失与ℓ₁正则化的鲁棒人脸识别

Robust face recognition based on the wing loss and the $\ell_1$ regularization

针对高遮挡、高损坏人脸图像下现有稀疏采样模型识别率下降的问题，提出wing约束稀疏编码模型及其加权版本，用ADMM求解，在多个人脸数据集上验证了其高识别率与强鲁棒性。

Human and Sheep Facial Landmarks Localisation by Triplet Interpolated Features

In this paper we present a method for localisation of facial landmarks on human and sheep. We introduce a new feature extraction scheme called triplet-interpolated feature used at each iteration of the cascaded shape regression framework. It is able to extract features from similar semantic location given an estimated shape, even when head pose variations are large and the facial landmarks are very sparsely distributed. Furthermore, we study the impact of training data imbalance on model performance and propose a training sample augmentation scheme that produces more initialisations for traini…

Performance analysis of weighted low rank model with sparse image histograms for face recognition under lowlevel illumination and occlusion

In a broad range of computer vision applications, the purpose of Low-rank matrix approximation (LRMA) models is to recover the underlying low-rank matrix from its degraded observation. The latest LRMA methods - Robust Principal Component Analysis (RPCA) resort to using the nuclear norm minimization (NNM) as a convex relaxation of the non-convex rank minimization. However, NNM tends to over-shrink the rank components and treats the different rank components equally, limiting its flexibility in practical applications. We use a more flexible model, namely the Weighted Schatten p-Norm Minimization…

Image Similarity Using Sparse Representation and Compression Distance

A new line of research uses compression methods to measure the similarity between signals. Two signals are considered similar if one can be compressed significantly when the information of the other is known. The existing compression-based similarity methods, although successful in the discrete one dimensional domain, do not work well in the context of images. This paper proposes a sparse representation-based approach to encode the information content of an image using information from the other image, and uses the compactness (sparsity) of the representation as a measure of its compressibilit…

Sparse Tensor-based Multiscale Representation for Point Cloud Geometry Compression

This study develops a unified Point Cloud Geometry (PCG) compression method through the processing of multiscale sparse tensor-based voxelized PCG. We call this compression method SparsePCGC. The proposed SparsePCGC is a low complexity solution because it only performs the convolutions on sparsely-distributed Most-Probable Positively-Occupied Voxels (MP-POV). The multiscale representation also allows us to compress scale-wise MP-POVs by exploiting cross-scale and same-scale correlations extensively and flexibly. The overall compression efficiency highly depends on the accuracy of estimated occ…

Image-to-Video Generation via 3D Facial Dynamics

We present a versatile model, FaceAnime, for various video generation tasks from still images. Video generation from a single face image is an interesting problem and usually tackled by utilizing Generative Adversarial Networks (GANs) to integrate information from the input face image and a sequence of sparse facial landmarks. However, the generated face images usually suffer from quality loss, image distortion, identity change, and expression mismatching due to the weak representation capacity of the facial landmarks. In this paper, we propose to "imagine" a face video from a single face imag…

3D Face Recognition with Sparse Spherical Representations

This paper addresses the problem of 3D face recognition using simultaneous sparse approximations on the sphere. The 3D face point clouds are first aligned with a novel and fully automated registration process. They are then represented as signals on the 2D sphere in order to preserve depth and geometry information. Next, we implement a dimensionality reduction process with simultaneous sparse approximations and subspace projection. It permits to represent each 3D face by only a few spherical functions that are able to capture the salient facial characteristics, and hence to preserve the discri…

Multi-view Face Analysis Based on Gabor Features

Facial analysis has attracted much attention in the technology for human-machine interface. Different methods of classification based on sparse representation and Gabor kernels have been widely applied in the fields of facial analysis. However, most of these methods treat face from a whole view standpoint. In terms of the importance of different facial views, in this paper, we present multi-view face analysis based on sparse representation and Gabor wavelet coefficients. To evaluate the performance, we conduct face analysis experiments including face recognition (FR) and face expression recogn…

从稀疏到浓密：无先验的多视角面部毛发级联重建

Wispy to Voluminous: Prior-free Multi-view Capture of Strand-level Facial Hair

提出从多视角图像自动重建面部毛发（胡须、眉毛等）的管线，将无结构3D高斯表示转换为显式曲线发丝，解决几何歧义，实现高保真发丝重建。

Detecting Face2Face Facial Reenactment in Videos

Visual content has become the primary source of information, as evident in the billions of images and videos, shared and uploaded on the Internet every single day. This has led to an increase in alterations in images and videos to make them more informative and eye-catching for the viewers worldwide. Some of these alterations are simple, like copy-move, and are easily detectable, while other sophisticated alterations like reenactment based DeepFakes are hard to detect. Reenactment alterations allow the source to change the target expressions and create photo-realistic images and videos. While …

A Sparse Representation of Complete Local Binary Pattern Histogram for Human Face Recognition

Human face recognition has been a long standing problem in computer vision and pattern recognition. Facial analysis can be viewed as a two-fold problem, namely (i) facial representation, and (ii) classification. So far, many face representations have been proposed, a well-known method is the Local Binary Pattern (LBP), which has witnessed a growing interest. In this respect, we treat in this paper the issues of face representation as well as classification in a novel manner. On the one hand, we use a variant to LBP, so-called Complete Local Binary Pattern (CLBP), which differs from the basic L…

Convolutional neural networks compression with low rank and sparse tensor decompositions

Convolutional neural networks show outstanding results in a variety of computer vision tasks. However, a neural network architecture design usually faces a trade-off between model performance and computational/memory complexity. For some real-world applications, it is crucial to develop models, which can be fast and light enough to run on edge systems and mobile devices. However, many modern architectures that demonstrate good performance don't satisfy inference time and storage limitation requirements. Thus, arises a problem of neural network compression to obtain a smaller and faster model, …

Masked Linear Regression for Learning Local Receptive Fields for Facial Expression Synthesis

Compared to facial expression recognition, expression synthesis requires a very high-dimensional mapping. This problem exacerbates with increasing image sizes and limits existing expression synthesis approaches to relatively small images. We observe that facial expressions often constitute sparsely distributed and locally correlated changes from one expression to another. By exploiting this observation, the number of parameters in an expression synthesis model can be significantly reduced. Therefore, we propose a constrained version of ridge regression that exploits the local and sparse struct…

Synthesizing Normalized Faces from Facial Identity Features

We present a method for synthesizing a frontal, neutral-expression image of a person's face given an input face photograph. This is achieved by learning to generate facial landmarks and textures from features extracted from a facial-recognition network. Unlike previous approaches, our encoding feature vector is largely invariant to lighting, pose, and facial expression. Exploiting this invariance, we train our decoder network using only frontal, neutral-expression photographs. Since these photographs are well aligned, we can decompose them into a sparse set of landmark points and aligned textu…

Efficient Data Compression for 3D Sparse TPC via Bicephalous Convolutional Autoencoder

Real-time data collection and analysis in large experimental facilities present a great challenge across multiple domains, including high energy physics, nuclear physics, and cosmology. To address this, machine learning (ML)-based methods for real-time data compression have drawn significant attention. However, unlike natural image data, such as CIFAR and ImageNet that are relatively small-sized and continuous, scientific data often come in as three-dimensional data volumes at high rates with high sparsity (many zeros) and non-Gaussian value distribution. This makes direct application of popul…

Face Aging Effect Simulation using Hidden Factor Analysis Joint Sparse Representation

Face aging simulation has received rising investigations nowadays, whereas it still remains a challenge to generate convincing and natural age-progressed face images. In this paper, we present a novel approach to such an issue by using hidden factor analysis joint sparse representation. In contrast to the majority of tasks in the literature that handle the facial texture integrally, the proposed aging approach separately models the person-specific facial properties that tend to be stable in a relatively long period and the age-specific clues that change gradually over time. It then merely tran…

EAST: Encoding-Aware Sparse Training for Deep Memory Compression of ConvNets

The implementation of Deep Convolutional Neural Networks (ConvNets) on tiny end-nodes with limited non-volatile memory space calls for smart compression strategies capable of shrinking the footprint yet preserving predictive accuracy. There exist a number of strategies for this purpose, from those that play with the topology of the model or the arithmetic precision, e.g. pruning and quantization, to those that operate a model agnostic compression, e.g. weight encoding. The tighter the memory constraint, the higher the probability that these techniques alone cannot meet the requirement, hence m…

HybridGS：利用双通道稀疏表示与点云编码器的高效高斯泼溅数据压缩

HybridGS: High-Efficiency Gaussian Splatting Data Compression using Dual-Channel Sparse Representation and Point Cloud Encoder

HybridGS提出结合紧凑生成与标准化点云编码的3DGS压缩框架，通过双通道稀疏表示和码率控制，在保持重建质量的同时显著提升编解码速度。

高效视频扩散与稀疏信息传输用于视频压缩

Efficient Video Diffusion with Sparse Information Transmission for Video Compression

本文提出Diff-SIT方法，通过稀疏时序编码模块和单步视频扩散模型提升视频压缩的感知质量和时间一致性，尤其在超低比特率下表现优异。

Accurate 3D Facial Geometry Prediction by Multi-Task, Multi-Modal, and Multi-Representation Landmark Refinement Network

This work focuses on complete 3D facial geometry prediction, including 3D facial alignment via 3D face modeling and face orientation estimation using the proposed multi-task, multi-modal, and multi-representation landmark refinement network (M$^3$-LRN). Our focus is on the important facial attributes, 3D landmarks, and we fully utilize their embedded information to guide 3D facial geometry learning. We first propose a multi-modal and multi-representation feature aggregation for landmark refinement. Next, we are the first to study 3DMM regression from sparse 3D landmarks and utilize multi-repre…

SparseGF: 一种具有高度意识的稀疏分割框架与上下文压缩，用于在城市到自然场景中实现鲁棒的地面过滤

SparseGF: A Height-Aware Sparse Segmentation Framework with Context Compression for Robust Ground Filtering Across Urban to Natural Scenes

本文提出SparseGF，通过上下文压缩模块、混合稀疏体素点网络架构和高度感知损失函数，解决大规模处理中的上下文细节矛盾和高大物体随机误分类问题，实现跨场景鲁棒的地面过滤。

Linear Disentangled Representation Learning for Facial Actions

Limited annotated data available for the recognition of facial expression and action units embarrasses the training of deep networks, which can learn disentangled invariant features. However, a linear model with just several parameters normally is not demanding in terms of training data. In this paper, we propose an elegant linear model to untangle confounding factors in challenging realistic multichannel signals such as 2D face videos. The simple yet powerful model does not rely on huge training data and is natural for recognizing facial actions without explicitly disentangling the identity. …

MesoNet: a Compact Facial Video Forgery Detection Network

This paper presents a method to automatically and efficiently detect face tampering in videos, and particularly focuses on two recent techniques used to generate hyper-realistic forged videos: Deepfake and Face2Face. Traditional image forensics techniques are usually not well suited to videos due to the compression that strongly degrades the data. Thus, this paper follows a deep learning approach and presents two networks, both with a low number of layers to focus on the mesoscopic properties of images. We evaluate those fast networks on both an existing dataset and a dataset we have constitut…

T-FAKE：合成用于面部关键点标注的热图像

T-FAKE: Synthesizing Thermal Images for Facial Landmarking

本文提出T-FAKE合成热面部数据集和RGB2Thermal损失函数，通过域自适应风格迁移生成带关键点标注的热图像，显著提升热图像面部关键点检测性能。

HGC-Avatar：面向可流式传输动态3D化身的分层高斯压缩

HGC-Avatar: Hierarchical Gaussian Compression for Streamable Dynamic 3D Avatars

本文提出HGC-Avatar分层高斯压缩框架，通过结构层、SMPL-X运动层及面部注意力实现动态3D化身的可流式传输、渐进解码与高质量渲染。

边缘视觉-语言模型的极端模型压缩：稀疏时间标记融合与自适应神经压缩

Extreme Model Compression for Edge Vision-Language Models: Sparse Temporal Token Fusion and Adaptive Neural Compression

本文提出稀疏时间标记融合和自适应神经压缩技术，实现边缘视觉-语言模型的高效压缩，提升准确率并降低延迟。

DiffCom：解耦稀疏先验引导的点云扩散压缩

DiffCom: Decoupled Sparse Priors Guided Diffusion Compression for Point Clouds

DiffCom提出一种由解耦稀疏先验引导的扩散压缩框架，通过双密度数据流和注意力引导去噪器，在低比特率下实现高质量点云重建，并在ShapeNet等数据集上取得更优率失真性能。

批Transformer架构：面向表情人脸识别的合成图像生成案例

Batch Transformer Architecture: Case of Synthetic Image Generation for Emotion Expression Facial Recognition

本文提出隐式稀疏风格的Batch Transformer架构，通过仅关注重要维度缩小编解码网络瓶颈，在带妆容遮挡的人脸识别合成图像生成任务中可提升有限原始数据集的多样性。