We introduce GoMAvatar, a novel approach for real-time, memory-efficient, high-quality animatable human modeling. GoMAvatar takes as input a single monocular video to create a digital avatar capable of re-articulation in new poses and real-time rendering from novel viewpoints, while seamlessly integrating with rasterization-based graphics pipelines. Central to our method is the Gaussians-on-Mesh representation, a hybrid 3D model combining rendering quality and speed of Gaussian splatting with geometry modeling and compatibility of deformable meshes. We assess GoMAvatar on ZJU-MoCap data and various YouTube videos. GoMAvatar matches or surpasses current monocular human modeling algorithms in rendering quality and significantly outperforms them in computational efficiency (43 FPS) while being memory-efficient (3.63 MB per subject).
We present animatable neural radiance fields (animatable NeRF) for detailed human avatar creation from monocular videos. Our approach extends neural radiance fields (NeRF) to the dynamic scenes with human movements via introducing explicit pose-guided deformation while learning the scene representation network. In particular, we estimate the human pose for each frame and learn a constant canonical space for the detailed human template, which enables natural shape deformation from the observation space to the canonical space under the explicit control of the pose parameters. To compensate for i…
Deblur-Avatar: Animatable Avatars from Motion-Blurred Monocular Videos
School of Artificial Intelligence and Automation, Huazhong University of Science and Technology（华中科技大学人工智能与自动化学院）；S-Lab for Advanced Intelligence, Nanyang Technological University（南洋理工大学先进智能实验室）；College of Computing and Data Science, Nanyang Technological University（南洋理工大学计算与数据科学学院）；SenseTime Research（商汤科技研究院）
Peking University（北京大学）；Microsoft Research Asia（微软亚洲研究院）；Tencent（腾讯）
We present a novel paradigm of building an animatable 3D human representation from a monocular video input, such that it can be rendered in any unseen poses and views. Our method is based on a dynamic Neural Radiance Field (NeRF) rigged by a mesh-based parametric 3D human model serving as a geometry proxy. Previous methods usually rely on multi-view videos or accurate 3D geometry information as additional inputs; besides, most methods suffer from degraded quality when generalized to unseen poses. We identify that the key to generalization is a good input embedding for querying dynamic NeRF: A …
ETH Zürich（苏黎世联邦理工学院）；Max Planck Institute for Intelligent Systems, Tübingen（马克斯·普朗克智能系统研究所（蒂宾根））；University of Tübingen（蒂宾根大学）
In this paper, we aim to create generalizable and controllable neural signed distance fields (SDFs) that represent clothed humans from monocular depth observations. Recent advances in deep learning, especially neural implicit representations, have enabled human shape reconstruction and controllable avatar generation from different sensor inputs. However, to generate realistic cloth deformations from novel input poses, watertight meshes or dense full-body scans are usually needed as inputs. Furthermore, due to the difficulty of effectively modeling pose-dependent cloth deformations for diverse …
Interactive Rendering of Relightable and Animatable Gaussian Avatars
State Key Lab of CAD & CG, Zhejiang University（浙江大学计算机辅助设计与图形学国家重点实验室）；UCL Centre for Artificial Intelligence, Department of Computer Science, University College London（伦敦大学学院计算机科学系人工智能中心）；Kahlert School of Computing, University of Utah（犹他大学Kahlert计算学院）
To address the ill-posed problem caused by partial observations in monocular human volumetric capture, we present AvatarCap, a novel framework that introduces animatable avatars into the capture pipeline for high-fidelity reconstruction in both visible and invisible regions. Our method firstly creates an animatable avatar for the subject from a small number (~20) of 3D scans as a prior. Then given a monocular RGB video of this subject, our method integrates information from both the image observation and the avatar prior, and accordingly recon-structs high-fidelity 3D textured models with dyna…
Facebook Reality Labs（Facebook现实实验室）；University of Southern California（南加州大学）
In this paper, we propose ARCH (Animatable Reconstruction of Clothed Humans), a novel end-to-end framework for accurate reconstruction of animation-ready 3D clothed humans from a monocular image. Existing approaches to digitize 3D humans struggle to handle pose variations and recover details. Also, they do not produce models that are animation ready. In contrast, ARCH is a learned pose-aware model that produces detailed 3D rigged full-body human avatars from a single unconstrained RGB image. A Semantic Space and a Semantic Deformation Field are created using a parametric 3D body estimator. The…
STAvatar: Soft Binding and Temporal Density Control for Monocular 3D Head Avatars Reconstruction
MAIS, Institute of Automation, Chinese Academy of Sciences（自动化研究所，中国科学院）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；CAIR, HKISI, Chinese Academy of Sciences（中国科学院计算技术研究所）；SCSE, FIE, M.U.S.T（M.U.S.T）
HRAvatar: High-Quality and Relightable Gaussian Head Avatar
Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院，清华大学）；International Digital Economy Academy (IDEA) Intern（国际数字经济发展学院（IDEA）实习生）
Topology-aware Human Avatars with Semantically-guided Gaussian Splatting
School of Computer Science, Wuhan University（武汉大学计算机学院）；School of Software Engineering, Xi’an Jiao Tong University（西安交通大学软件学院）；AI Institute, Shanghai Jiao Tong University（上海交通大学人工智能研究院）；Huazhong University of Science and Technology（华中科技大学）
STG-Avatar: Animatable Human Avatars via Spacetime Gaussian
School of Information and Communication Engineering, Dalian University of Technology（信息与通信工程学院，大连理工大学）；School of Information Science and Technology, Fudan University（信息科学与技术学院，复旦大学）；Faculty of Science and Technology, University of Macau（科学与技术学院，澳门大学）；University of Zaragoza（阿拉瓦大学）；Hong Kong University of Science and Technology (Guangzhou)（香港科学与技术大学（广州））
