How to Build Digital Humans? From Priors to Photorealistic Avatars
Meta;Technical University of Munich(慕尼黑技术大学);Google(谷歌);Tsinghua University(清华大学);Max Planck Institute for Intelligent Systems(智能系统马克斯·普朗克研究所);NVIDIA;Technical University of Darmstadt(达姆施塔特技术大学);ETH Zurich(苏黎世联邦理工学院)
Wojciech Zielonka, Tobias Kirschstein, Timo Bolkart, Simon Giebenhain, Vanessa Sklyarova, Xiang Deng, Donglai Xiang, Shunsuke Saito, Yebin Liu, Matthias Niessner, Justus Thies
This state-of-the-art report provides an overview of controllable 3D human avatar creation. We describe current 3D avatar systems, which typically consist of three stages: (i) learning priors of human appearance and motion, (ii) creating a personalized avatar, and (iii) animating the avatar. To limit the scope, we focus on the prior learning and avatar creation stages. We define current avatar representations and introduce a taxonomy that categorizes existing work along multiple axes, including body regions and employed priors. We review methods for full-body and head avatars, as well as layered representations that decompose the body into components such as hands, hair, and garments. Finally, we outline common underlying principles, reference key literature for newcomers, and discuss open challenges and future research directions.
SMPLX-Lite: A Realistic and Drivable Avatar Benchmark with Rich Geometry and Texture Annotations
Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院);Prometheus Vision Technology Co., Ltd.(普罗米修斯视觉科技有限公司);Beijing University of Aeronautics and Astronautics(北京航空航天大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州))
DEGAS: Detailed Expressions on Full-Body Gaussian Avatars
The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Prometheus Vision Technology Co., Ltd.(天视智能科技有限公司);The Hong Kong University of Science and Technology(香港科技大学);Swinburne University of Technology(斯威本科技大学)
WonderHuman: Hallucinating Unseen Parts in Dynamic 3D Human Reconstruction
Department of Computer Science, The University of Texas at Dallas(德克萨斯大学达拉斯分校计算机科学系);Computer Graphics Group, The University of Hong Kong(香港大学计算机图形组);School of Engineering, The Hong Kong University of Science and Technology(香港科学与技术大学工程学院);Guangdong Provincial/Zhuhai Key Laboratory of IRADS, Beijing Normal-Hong Kong Baptist University(广东/珠海IRADS重点实验室,北京师范大学-香港 Baptist大学);Department of Computer Science & Engineering, Texas A&M University(德克萨斯A&M大学计算机科学与工程系)
SEGA: Drivable 3D Gaussian Head Avatar from a Single Image
Tsinghua Shenzhen International Graduate School(清华大学深圳国际研究生院);ByteDance(字节跳动);Pengcheng Lab(鹏城实验室);Department of Automation, Tsinghua University(清华大学自动化系)
DELIFFAS: Deformable Light Fields for Fast Avatar Synthesis
University of North Carolina at Chapel Hill(北卡罗来纳大学教堂山分校);University of Pennsylvania(宾夕法尼亚大学);Max Planck Institute for Informatics(马克斯·普朗克信息学研究所);Saarbrücken Research Center for Visual Computing, Interaction and AI(萨尔布吕肯视觉计算、交互与人工智能研究中心)
SplattingAvatar: Realistic Real-Time Human Avatars with Mesh-Embedded Gaussian Splatting
The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Prometheus Vision Technology Co., Ltd.(天演智视科技有限公司);The Hong Kong University of Science and Technology(香港科技大学)
Envisioning a Next Generation Extended Reality Conferencing System with Efficient Photorealistic Human Rendering
University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);University of Miami(迈阿密大学);Arizona State University(亚利桑那州立大学);Futurewei Technologies(华为未来技术研究院)
IDOL: Instant Photorealistic 3D Human Creation from a Single Image
Nanjing University(南京大学);Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院);Tsinghua University(清华大学);Tencent(腾讯);Shenzhen University of Advanced Technology(深圳理工大学)