提出Gaussian Eigen Models (GEM)，通过蒸馏高质量CNN模型构建线性特征基，生成轻量且高质量的头部虚拟形象，实现单RGB图像输入的表情预测，在视觉质量和泛化性上优于现有3D avatar方法。
摘要：
当前个性化的神经头部虚拟形象面临一个权衡：轻量级模型缺乏细节和真实感，而高质量、可动画化的虚拟形象需要大量计算资源，使其不适用于普通设备。为解决这一问题，我们提出了Gaussian Eigen Models (GEM)，它提供高质量、轻量且易于控制的头部虚拟形象。GEM利用3D Gaussian primitives表示外观，并结合Gaussian splatting进行渲染。基于基于网格的3D morphable face models (3DMM)的成功，我们将GEM定义为线性特征基的集合，用于表示特定主体的头部外观。具体而言，我们构建线性基来表示3D Gaussians的位置、缩放、旋转和不透明度。这使我们能够通过基向量的线性组合高效生成特定头部形状的Gaussian primitives，仅需包含相应系数的低维参数向量。我们提出通过蒸馏高质量的、基于计算密集型CNN的Gaussian avatar模型来构建这些线性基 (GEM)，这些模型能生成表情相关的外观变化，如皱纹。这些高质量模型在主体的多视角视频上训练，并通过一系列主成分分析进行蒸馏。一旦获得表示特定人类可动画外观空间的基，我们便学习一个回归器，以单个RGB图像作为输入，预测与所示面部表情对应的低维参数向量。在一系列实验中，我们将GEM的自重演和跨人重演结果与最先进的3D avatar方法进行比较，证明GEM具有更高的视觉质量和对新表情的更好泛化能力。
英文摘要：
Current personalized neural head avatars face a trade-off: lightweight models lack detail and realism, while high-quality, animatable avatars require significant computational resources, making them unsuitable for commodity devices. To address this gap, we introduce Gaussian Eigen Models (GEM), which provide high-quality, lightweight, and easily controllable head avatars. GEM utilizes 3D Gaussian primitives for representing the appearance combined with Gaussian splatting for rendering. Building on the success of mesh-based 3D morphable face models (3DMM), we define GEM as an ensemble of linear eigenbases for representing the head appearance of a specific subject. In particular, we construct linear bases to represent the position, scale, rotation, and opacity of the 3D Gaussians. This allows us to efficiently generate Gaussian primitives of a specific head shape by a linear combination of the basis vectors, only requiring a low-dimensional parameter vector that contains the respective coefficients. We propose to construct these linear bases (GEM) by distilling high-quality compute-intense CNN-based Gaussian avatar models that can generate expression-dependent appearance changes like wrinkles. These high-quality models are trained on multi-view videos of a subject and are distilled using a series of principal component analyses. Once we have obtained the bases that represent the animatable appearance space of a specific human, we learn a regressor that takes a single RGB image as input and predicts the low-dimensional parameter vector that corresponds to the shown facial expression. In a series of experiments, we compare GEM's self-reenactment and cross-person reenactment results to state-of-the-art 3D avatar methods, demonstrating GEM's higher visual quality and better generalization to new expressions.
GaussianEmoTalker: Real-Time Emotional Talking Head Synthesis with Audio-Driven and Blendshape-Based 3D Gaussian Splatting
PCA Lab, Key Lab of Intelligent Perception and Systems for High-Dimensional Information of Ministry of Education, School of Computer Science and Engineering, Nanjing University of Science and Technology（南京理工大学计算机科学与工程学院，教育部高维信息智能感知与系统重点实验室，PCA实验室）；Tsientang Institute for Advanced Study（浙江大学前沿交叉研究院）；School of Intelligence Science and Technology, Nanjing University（南京大学智能科学与技术学院）
MixedGaussianAvatar: Realistically and Geometrically Accurate Head Avatar via Mixed 2D-3D Gaussians
University of the Chinese Academy of Sciences（中国科学院大学）；Peking University（北京大学）；Nankai University（南开大学）；Tsinghua University（清华大学）；Intel Labs China（英特尔中国实验室）
HRAvatar: High-Quality and Relightable Gaussian Head Avatar
Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院，清华大学）；International Digital Economy Academy (IDEA) Intern（国际数字经济发展学院（IDEA）实习生）
Generating Editable Head Avatars with 3D Gaussian GANs
School of Computer Science and Engineering, Beihang University（北京航空航天大学计算机科学与工程学院）；School of Artificial Intelligence, Beihang University（北京航空航天大学人工智能学院）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）
EmbedTalk: Talking Head Synthesis using Gaussian Embeddings
School of Computer Science, University of Leeds（利兹大学计算机科学学院）；Leeds Institute of Medical Education, School of Medicine, University of Leeds（利兹大学医学教育研究所）
TalkingGaussian: Structure-Persistent 3D Talking Head Synthesis via Gaussian Splatting
Beihang University（北京航空航天大学）；Institute of Semiconductors, Chinese Academy of Sciences（中国科学院半导体研究所）；Griffith University（格里菲斯大学）；RIKEN AIP（理化学研究所人工智能项目）；The University of Tokyo（东京大学）
SyncTalk++: High-Fidelity and Efficient Synchronized Talking Heads Synthesis Using Gaussian Splatting
School of Information, Renmin University of China（中国人民大学信息学院）；School of Information and Communication Engineering, Beijing University of Posts and Telecommunications（北京邮电大学信息与通信工程学院）；Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院航天信息研究所）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Institute for AI Industry Research, Tsinghua University（清华大学人工智能产业研究院）；School of Economics and Management, Tsinghua University（清华大学经济管理学院）；Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing, School of Artificial Intelligence, Beihang University（北京航空航天大学人工智能学院）
Monocular and Generalizable Gaussian Talking Head Animation
South China University of Technology（华南理工大学）；Technical University of Munich（慕尼黑工业大学）；Pazhou Laboratory（琶洲实验室）；Guangdong University of Technology（广东工业大学）；The Hong Kong Polytechnic University（香港理工大学）
AvatarBack: Back-Head Generation for Complete 3D Avatars from Front-View Images
College of Electrical Engineering and Automation, Shandong University of Science and Technology（山东科技大学电气工程与自动化学院）；Department of Data Science and Artificial Intelligence, Auckland University of Technology（奥克兰大学数据科学与人工智能系）；College of Computer Science and Engineering, Shandong University of Science and Technology（山东科技大学计算机科学与工程学院）
Toward Fine-Grained Facial Control in 3D Talking Head Generation
School of Cyber Science and Engineering, Southeast University（东南大学计算机科学与工程学院）；Lee Kong Chian School of Medicine, Nanyang Technological University（南洋理工大学李科金医学院）；School of Remote Sensing and Information Engineering, Wuhan University（武汉大学遥感与信息工程学院）；Purple Mountain Laboratories, Nanjing（南京紫金山实验室）；Engineering Research Center of Blockchain Application, Supervision And Management (Southeast University), Ministry of Education（教育部区块链应用、监督与管理工程研究中心（东南大学））；Department of Computer and Information Science, University of Macau（澳门大学计算机与信息科学系）；Department of Computer Science and Engineering, The Hong Kong University of Science and Technology（香港科技大学计算机科学与工程系）
SEGA: Drivable 3D Gaussian Head Avatar from a Single Image
Tsinghua Shenzhen International Graduate School（清华大学深圳国际研究生院）；ByteDance（字节跳动）；Pengcheng Lab（鹏城实验室）；Department of Automation, Tsinghua University（清华大学自动化系）
SketchFaceGS: Real-Time Sketch-Driven Face Editing and Generation with Gaussian Splatting
Shandong Technology and Business University（山东科技与商务大学）；Nanchang Hangkong University（南昌航空大学）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；University of New South Wales（新南威尔士大学）
ETH Zürich（苏黎世联邦理工学院）；Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；Max Planck Institute for Informatics（马克斯·普朗克信息研究所）；Tübingen AI Center（图宾根人工智能中心）；Technical University of Munich（慕尼黑工业大学）；Technical University of Darmstadt（达姆施塔特工业大学）；Microsoft（微软）
Audio-Plane: Audio Factorization Plane Gaussian Splatting for Real-Time Talking Head Synthesis
School of Electrical and Electronic Engineering, Nanyang Technological University（南洋理工大学电子与电气工程学院）；John A. Paulson School of Engineering and Applied Sciences at Harvard University（哈佛大学约翰·A·保罗森工程与应用科学学院）；PhiGent Robotics（PhiGent机器人技术公司）；Beijing National Research Center for Information Science and Technology (BNRist) and the Department of Automation, Tsinghua University（北京信息科学与技术国家研究中心（BNRist）和清华大学自动化系）
GaussianHead: High-fidelity Head Avatars with Learnable Gaussian Derivation
School of Artificial Intelligence, Nanjing University of Posts and Telecommunications（南京邮电大学人工智能学院）；BNRist, Tsinghua University（清华大学BNRist）；Department of Computer and Information Science, University of Macau（澳门大学计算机与信息科学系）
MGAvatar: Mesh-Bound Gaussians for Head Avatar Geometry and Appearance Modeling
Beijing Normal-Hong Kong Baptist University（北京师范大学-香港浸会大学联合国际学院）；Xiamen University（厦门大学）；The University of Texas at Dallas（德克萨斯大学达拉斯分校）；Jimei University（集美大学）
