AIM 2024 Sparse Neural Rendering Challenge: Methods and Results
University of Würzburg（维尔茨堡大学）
Michal Nazarczuk, Sibi Catley-Chandar, Thomas Tanay, Richard Shaw, Eduardo Pérez-Pellitero, Radu Timofte, Xing Yan, Pan Wang, Yali Guo, Yongxin Wu, Youcheng Cai… 展开作者
Michal Nazarczuk, Sibi Catley-Chandar, Thomas Tanay, Richard Shaw, Eduardo Pérez-Pellitero, Radu Timofte, Xing Yan, Pan Wang, Yali Guo, Yongxin Wu, Youcheng Cai, Yanan Yang, Junting Li, Yanghong Zhou, P. Y. Mok, Zongqi He, Zhe Xiao, Kin-Chung Chan, Hana Lebeta Goshu, Cuixin Yang, Rongkang Dong, Jun Xiao, Kin-Man Lam, Jiayao Hao, Qiong Gao, Yanyan Zu, Junpei Zhang, Licheng Jiao, Xu Liu, Kuldeep Purohit
备注：
Part of Advances in Image Manipulation workshop at ECCV 2024
This paper reviews the challenge on Sparse Neural Rendering that was part of the Advances in Image Manipulation (AIM) workshop, held in conjunction with ECCV 2024. This manuscript focuses on the competition set-up, the proposed methods and their respective results. The challenge aims at producing novel camera view synthesis of diverse scenes from sparse image observations. It is composed of two tracks, with differing levels of sparsity; 3 views in Track 1 (very sparse) and 9 views in Track 2 (sparse). Participants are asked to optimise objective fidelity to the ground-truth images as measured via the Peak Signal-to-Noise Ratio (PSNR) metric. For both tracks, we use the newly introduced Sparse Rendering (SpaRe) dataset and the popular DTU MVS dataset. In this challenge, 5 teams submitted final results to Track 1 and 4 teams submitted final results to Track 2. The submitted models are varied and push the boundaries of the current state-of-the-art in sparse neural rendering. A detailed description of all models developed in the challenge is provided in this paper.
Few-shot Neural Human Performance Rendering from Sparse RGBD Videos
ShanghaiTech University（上海科技大学）；Shanghai Institute of Microsystem and Information Technology, Chinese Academy of Sciences（中国科学院上海微系统与信息技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）
Department of Electronic and Computer Engineering, The Hong Kong University of Science and Technology（电子与计算机工程系，香港科学与技术大学）；Microelectronics Thrust, The Hong Kong University of Science and Technology (GZ)（微电子领域，香港科学与技术大学（GZ））
GAINS: Gaussian-based Inverse Rendering from Sparse Multi-View Captures
University Kaiserslautern-Landau（凯撒斯劳滕-兰道大学）；German Research Center for Artificial Intelligence（德国人工智能研究中心）；University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；College of William & Mary（威廉与玛丽学院）
RDG-GS: Relative Depth Guidance with Gaussian Splatting for Real-time Sparse-View 3D Rendering
College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；College of Biomedical Engineering and Instrument Science, Zhejiang University（浙江大学生物医学工程与仪器科学学院）；ZJU-UIUC Institute, Zhejiang University（浙江大学-伊利诺伊大学厄巴纳-香槟分校联合学院）
Holoported Characters: Real-time Free-viewpoint Rendering of Humans from Sparse RGB Cameras
Max Planck Institute for Informatics（马克斯·普朗克信息学研究所）；Saarland University（萨尔大学）；Saarbrücken Research Center for Visual Computing, Interaction and AI（萨尔布吕肯视觉计算、交互与人工智能研究中心）
Physically Plausible Human-Object Rendering from Sparse Views via 3D Gaussian Splatting
College of Computer Science, Zhejiang University（浙江大学计算机科学与技术学院）；Department of Computer Science and Engineering, The Hong Kong University of Science and Technology（香港科技大学计算机科学与工程系）
VAnim: Rendering-Aware Sparse State Modeling for Structure-Preserving Vector Animation
School of Software, Beihang University, Beijing, China（北京航空航天大学软件学院）；Department of Computer Science, The University of Hong Kong, Hong Kong, China（香港大学计算机科学系）；College of Computer Science and Technology, Zhejiang University, Hangzhou, China（浙江大学计算机科学与技术学院）
Holi-Spatial 360: Generative Reconstruction from Sparse Panoramas for Spatial Rendering and Intelligence
The Hong Kong University of Science and Technology（香港科技大学）；ShanghaiTech University（上海科技大学）；Tencent（腾讯）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；The University of Tokyo（东京大学）；Northwestern Polytechnical University（西北工业大学）；Shanghai Jiao Tong University（上海交通大学）
本文提出 Holi-Spatial 360 框架，将房产平台稀疏全景图转为带注释多房间 3D 环境，构建含海量数据的 Holi360 基准，可提升 VLM 空间推理与重建整理质量。
Rendering-Oriented 3D Point Cloud Attribute Compression using Sparse Tensor-based Transformer
School of Telecommunication Engineering, Xidian University（西安电子科技大学通信工程学院）；City University of Hong Kong（香港城市大学）；School of Electronics and Information, Northwestern Polytechnical University（西北工业大学电子信息学院）；School of Engineering, Royal Melbourne Institute of Technology（皇家墨尔本理工大学工程学院）
HumanNeRF: Efficiently Generated Human Radiance Field from Sparse Inputs
ShanghaiTech University（上海科技大学）；Huazhong University of Science and Technology（华中科技大学）；DGene；Shanghai Engineering Research Center of Intelligent Vision and Imaging（上海智能视觉与成像工程技术研究中心）
