我们提出了PRTGS(Precomputed Radiance Transfer of Gaussian Splats),一种针对低频光照环境下高斯泼溅的实时高质量重光照方法,通过预计算3D高斯泼溅的辐射传输来捕获柔和阴影和相互反射。已有研究表明,3D高斯泼溅(3DGS)在动态光照场景中的效率优于神经场。然而,当前基于3DGS的重光照方法仍难以在动态光照下实时计算高质量的阴影和间接光照,导致渲染结果不真实。我们通过预计算复杂传输函数(如阴影)所需的昂贵传输模拟来解决这一问题,所得传输函数表示为每个高斯泼溅的稠密向量或矩阵集合。我们引入了分别针对训练和渲染阶段的独特预计算方法,以及针对3D高斯泼溅的独特光线追踪和间接光照预计算技术,以加速训练速度并计算与环境光相关的精确间接光照。实验分析表明,我们的方法在保持有竞争力的训练时间的同时达到了最先进的视觉质量,并能在1080p分辨率下对动态光和相对复杂的场景实现高质量实时(30+ fps)重光照。
英文摘要:
We proposed Precomputed RadianceTransfer of GaussianSplats (PRTGS), a real-time high-quality relighting method for Gaussian splats in low-frequency lighting environments that captures soft shadows and interreflections by precomputing 3D Gaussian splats' radiance transfer. Existing studies have demonstrated that 3D Gaussian splatting (3DGS) outperforms neural fields' efficiency for dynamic lighting scenarios. However, the current relighting method based on 3DGS still struggles to compute high-quality shadow and indirect illumination in real time for dynamic light, leading to unrealistic rendering results. We solve this problem by precomputing the expensive transport simulations required for complex transfer functions like shadowing, the resulting transfer functions are represented as dense sets of vectors or matrices for every Gaussian splat. We introduce distinct precomputing methods tailored for training and rendering stages, along with unique ray tracing and indirect lighting precomputation techniques for 3D Gaussian splats to accelerate training speed and compute accurate indirect lighting related to environment light. Experimental analyses demonstrate that our approach achieves state-of-the-art visual quality while maintaining competitive training times and allows high-quality real-time (30+ fps) relighting for dynamic light and relatively complex scenes at 1080p resolution.
ROSGS: Relightable Outdoor Scenes With Gaussian Splatting
School of Information Science and Technology, North China University of Technology(信息科学与技术学院,中国北方工业大学);Beijing Key Laboratory of Mobile Computing and Pervasive Device, Institute of Computing Technology, Chinese Academy of Sciences(移动计算与普适设备北京市重点实验室,中国科学院计算技术研究所);University of Chinese Academy of Sciences(中国科学院大学);School of Computer Science and Informatics, Cardiff University(计算机科学与信息学院,卡迪夫大学)
LumiMotion: Improving Gaussian Relighting with Scene Dynamics
Warsaw University of Technology(华沙理工大学);Sano Centre for Computational Medicine(Sano计算医学中心);Inst. for Biomedical Informatics, Univ. Hospital of Cologne(科隆大学医院生物医学信息学研究所);University of Cologne(科隆大学);CMMC Cologne(科隆CMMC);Jagiellonian University(雅盖隆大学);IDEAS Research Institute(IDEAS研究所);Microsoft(微软)
GeoSplatting: Towards Geometry Guided Gaussian Splatting for Physically-based Inverse Rendering
School of Intelligence Science and Technology, Peking University(北京大学智能科学与技术学院);School of Computer Science and Engineering, Sun Yat-sen University(中山大学计算机科学与工程学院);Wangxuan Institute of Computer Technology, Peking University(北京大学王轩计算机技术研究所);State Key Laboratory of Multimedia Information Processing, Peking University(北京大学多媒体信息处理国家重点实验室);State Key Laboratory of General Artificial Intelligence, Peking University(北京大学通用人工智能国家重点实验室)
GS-ROR$^2$: Bidirectional-guided 3DGS and SDF for Reflective Object Relighting and Reconstruction
VCIP, College of Computer Science, Nankai University(VCIP,计算机科学学院,南开大学);School of Intelligence Science and Technology, Nanjing University(智能科学与技术学校,南京大学)
GS-IR: 3D Gaussian Splatting for Inverse Rendering
South China University of Technology(华南理工大学);Tencent AI Lab(腾讯人工智能实验室);School of Data Science, The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳)数据科学学院)
GUS-IR: Gaussian Splatting with Unified Shading for Inverse Rendering
School of Electronic and Information Engineering, South China University of Technology(华南理工大学电子与信息工程学院);Australian National University(澳大利亚国立大学);School of Data Science, The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳)数据科学学院);VIVO
Large Material Gaussian Model for Relightable 3D Generation
Tsinghua Shenzhen International Graduate School(清华大学深圳国际研究生院);The University of Hong Kong(香港大学);LIGHTSPEED;The Hong Kong University of Science and Technology (Guangzhou)(香港科学与技术大学(广州))
HRAvatar: High-Quality and Relightable Gaussian Head Avatar
Tsinghua Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院,清华大学);International Digital Economy Academy (IDEA) Intern(国际数字经济发展学院(IDEA)实习生)
GAINS: Gaussian-based Inverse Rendering from Sparse Multi-View Captures
University Kaiserslautern-Landau(凯撒斯劳滕-兰道大学);German Research Center for Artificial Intelligence(德国人工智能研究中心);University of North Carolina at Chapel Hill(北卡罗来纳大学教堂山分校);College of William & Mary(威廉与玛丽学院)