Virtual production (VP) use LED walls to provide both background imagery and image-based lighting. While this enables on-set compositing, it couples lighting to background and scene appearance, limiting flexibility for downstream editing. In addition, inverse rendering conventionally relies on physically-based rendering to estimates 3D geometry and lighting, using environment maps. However, these maps are typically low-resolution and assume far-field lighting. In VP, with near-field and high-resolution image-based lighting, this can lead to inaccuracies and introduce complexities when editing. Addressing this, we propose a VP-specific framework for 3D reconstruction and relighting using Gaussian Splatting. This uses the known background imagery to condition the relighting process. This avoids relying on environment maps and reduces compositing to a background-image editing task. To realize our framework, we introduce a process (and associated dataset) that captures real VP scenes under varying background content and illumination conditions. This data is used to decompose a 3D scene into fixed appearance and variable lighting components. The variable lighting process simulates light transport by parameterizing each primitive with a UV coordinate, intensity value and resolution modifier. Using mipmaps, these directly sample the background texture in image space - implicitly capturing reflections and refractions without physically-based rendering. Combined with the fixed appearance component, this allows us to render relit scenes using a Gaussian Splatting rasterizer. Compared to baselines, our approach achieves higher-quality 3D reconstruction and controllable relighting. The method is efficient (<3 GB RAM, <5 GB VRAM, <2 hours training, ~35 FPS) and supports rendering useful arbitrary output variables including depth, lighting intensity, lighting color, and unlit renders.
An Illumination-Robust Feature Extractor Augmented by Relightable 3D Reconstruction
College of Engineering, Peking University（北京大学工学院）；Institute for Artificial Intelligence, Peking University（北京大学人工智能研究院）；Beijing Engineering Research Center of Intelligent Rehabilitation Engineering（北京智能康复工程工程研究中心）；Department of Electronic and Computer Engineering, Hong Kong University of Science and Technology（香港科技大学电子与计算机工程系）；University of Health and Rehabilitation Sciences, Qingdao（青岛健康与康复科学大学）
Max Planck Institute for Informatics（马克斯·普朗克信息学研究所）；Institute of Software, Chinese Academy of Sciences（中国科学院软件研究所）
Learning neural radiance fields of a scene has recently allowed realistic novel view synthesis of the scene, but they are limited to synthesize images under the original fixed lighting condition. Therefore, they are not flexible for the eagerly desired tasks like relighting, scene editing and scene composition. To tackle this problem, several recent methods propose to disentangle reflectance and illumination from the radiance field. These methods can cope with solid objects with opaque surfaces but participating media are neglected. Also, they take into account only direct illumination or at m…
ROSGS: Relightable Outdoor Scenes With Gaussian Splatting
School of Information Science and Technology, North China University of Technology（信息科学与技术学院，中国北方工业大学）；Beijing Key Laboratory of Mobile Computing and Pervasive Device, Institute of Computing Technology, Chinese Academy of Sciences（移动计算与普适设备北京市重点实验室，中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；School of Computer Science and Informatics, Cardiff University（计算机科学与信息学院，卡迪夫大学）
PR-ENDO: Physically Based Relightable Gaussian Splatting for Endoscopy
Sano Centre for Computional Medicine, Poland（波兰萨诺计算医学中心）；Warsaw University of Technology, Poland（波兰华沙理工大学）；Doctoral School of Exact and Natural Sciences, Jagiellonian University, Poland（杰齐尔大学精确与自然科学博士学院）；Faculty of Mathematics and Computer Science, Jagiellonian University, Poland（杰齐尔大学数学与计算机科学学院）；University of Verona, Italy（意大利威尼斯大学）；IDEAS Research Institute, Poland（波兰IDEAS研究所）
R-PGA: Robust Physical Adversarial Camouflage Generation via Relightable 3D Gaussian Splatting
School of Cyber Science and Technology, Shenzhen Campus, Sun Yat-sen University（中山大学深圳校区网络空间安全学院）；School of Computing, National University of Singapore（新加坡国立大学计算机学院）；School of Intelligent Systems Engineering, Shenzhen Campus, Sun Yat-sen University（中山大学深圳校区智能工程学院）
NeuS-PIR: Learning Relightable Neural Surface using Pre-Integrated Rendering
Baidu Research（百度研究院）；King Abdullah University of Science and Technology（阿卜杜拉国王科技大学）；Paul G. Allen School of Computer Science, University of Washington（华盛顿大学保罗 G. 艾伦计算机科学学院）；Institute of Information Engineering, Chinese Academy of Science（中国科学院信息工程研究院）
Large Material Gaussian Model for Relightable 3D Generation
Tsinghua Shenzhen International Graduate School（清华大学深圳国际研究生院）；The University of Hong Kong（香港大学）；LIGHTSPEED；The Hong Kong University of Science and Technology (Guangzhou)（香港科学与技术大学（广州））
HRAvatar: High-Quality and Relightable Gaussian Head Avatar
Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院，清华大学）；International Digital Economy Academy (IDEA) Intern（国际数字经济发展学院（IDEA）实习生）
Moving Light Adaptive Colonoscopy Reconstruction via Illumination-Attenuation-Aware 3D Gaussian Splatting
Wuhan National Laboratory for Optoelectronics, Huazhong University of Science and Technology（武汉光电国家研究中心，华中科技大学）；School of Computer Science, Wuhan University（武汉大学计算机学院）；Wuhan United Imaging Healthcare Surgical Technology Co., Ltd（武汉联影医疗手术技术有限公司）
Continuous Splatting meets Retinex: Continuous Gaussian Splatting and Implicit Reflectance Modeling for Low-Light Image Enhancement
College of Mechanical and Vehicle Engineering, Chongqing University（重庆大学机械与运载工程学院）；School of Engineering Science, University of Science and Technology of China（中国科学技术大学工程科学学院）；National Innovation Center of Intelligent and Connected Vehicles（国家智能网联汽车创新中心）；School of Vehicle and Mobility, Tsinghua University（清华大学车辆与运载学院）
Endo-4DGX: Robust Endoscopic Scene Reconstruction and Illumination Correction with Gaussian Splatting
Department of Electronic Engineering, The Chinese University of Hong Kong (CUHK), Hong Kong SAR, China（香港中文大学电子工程系）；Shun Hing Institute of Advanced Engineering, CUHK, Hong Kong SAR, China（CUHK先进工程研究所）；City University of Hong Kong, Hong Kong, China（香港城市大学）；The University of Sydney, Sydney, NSW, Australia（悉尼大学）；Centre for Artificial Intelligence（人工智能中心）
Interactive Rendering of Relightable and Animatable Gaussian Avatars
State Key Lab of CAD & CG, Zhejiang University（浙江大学计算机辅助设计与图形学国家重点实验室）；UCL Centre for Artificial Intelligence, Department of Computer Science, University College London（伦敦大学学院计算机科学系人工智能中心）；Kahlert School of Computing, University of Utah（犹他大学Kahlert计算学院）
Light 'em Up: Enabling Few-Shot Low-Light 3D Gaussian Splatting with Multi-Scale Explicit Retinex Illumination Decoupling
Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院航空航天信息研究所）；Key Laboratory of Target Cognition and Application Technology (TCAT)（目标认知与应用技术重点实验室）；University of Chinese Academy of Sciences（中国科学院大学）；School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences（中国科学院大学电子电气与通信工程学院）；Shanghai Jiao Tong University（上海交通大学）
Physically Inspired Gaussian Splatting for HDR Novel View Synthesis
Department of Electrical and Computer Engineering, Northeastern University（东北大学电气与计算机工程系）；Khoury College of Computer Science, Northeastern University（东北大学Khoury计算机科学学院）
MaterialRefGS: Reflective Gaussian Splatting with Multi-view Consistent Material Inference
School of Software, Tsinghua University, Beijing, China（清华大学软件学院）；Center for AI and Robotics (CAIR), NYU Abu Dhabi, UAE（纽约大学阿布扎比分校人工智能与机器人中心）；Department of Computer Science, Wayne State University, Detroit, USA（韦恩州立大学计算机科学系）
MedGS: Gaussian Splatting for Multi-Modal 3D Medical Imaging
Jagiellonian University, Poland（雅盖隆大学，波兰）；Warsaw University of Technology, Poland（华沙理工大学，波兰）；IDEAS Research Institute, Poland（IDEAS研究所，波兰）；Military Institute of Medicine, Poland（军事医学研究所，波兰）
