Synthesizing NeRFs under arbitrary lighting has become a seminal problem in the last few years. Recent efforts tackle the problem via the extraction of physically-based parameters that can then be rendered under arbitrary lighting, but they are limited in the range of scenes they can handle, usually mishandling glossy scenes. We propose RRM, a method that can extract the materials, geometry, and environment lighting of a scene even in the presence of highly reflective objects. Our method consists of a physically-aware radiance field representation that informs physically-based parameters, and an expressive environment light structure based on a Laplacian Pyramid. We demonstrate that our contributions outperform the state-of-the-art on parameter retrieval tasks, leading to high-fidelity relighting and novel view synthesis on surfacic scenes.
ROSGS: Relightable Outdoor Scenes With Gaussian Splatting
School of Information Science and Technology, North China University of Technology（信息科学与技术学院，中国北方工业大学）；Beijing Key Laboratory of Mobile Computing and Pervasive Device, Institute of Computing Technology, Chinese Academy of Sciences（移动计算与普适设备北京市重点实验室，中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；School of Computer Science and Informatics, Cardiff University（计算机科学与信息学院，卡迪夫大学）
Max Planck Institute for Informatics（马克斯·普朗克信息学研究所）；Institute of Software, Chinese Academy of Sciences（中国科学院软件研究所）
Learning neural radiance fields of a scene has recently allowed realistic novel view synthesis of the scene, but they are limited to synthesize images under the original fixed lighting condition. Therefore, they are not flexible for the eagerly desired tasks like relighting, scene editing and scene composition. To tackle this problem, several recent methods propose to disentangle reflectance and illumination from the radiance field. These methods can cope with solid objects with opaque surfaces but participating media are neglected. Also, they take into account only direct illumination or at m…
Interactive Rendering of Relightable and Animatable Gaussian Avatars
State Key Lab of CAD & CG, Zhejiang University（浙江大学计算机辅助设计与图形学国家重点实验室）；UCL Centre for Artificial Intelligence, Department of Computer Science, University College London（伦敦大学学院计算机科学系人工智能中心）；Kahlert School of Computing, University of Utah（犹他大学Kahlert计算学院）
NeuS-PIR: Learning Relightable Neural Surface using Pre-Integrated Rendering
Baidu Research（百度研究院）；King Abdullah University of Science and Technology（阿卜杜拉国王科技大学）；Paul G. Allen School of Computer Science, University of Washington（华盛顿大学保罗 G. 艾伦计算机科学学院）；Institute of Information Engineering, Chinese Academy of Science（中国科学院信息工程研究院）
Max Planck Institute for Informatics（马克斯·普朗克信息学研究所）；MIT（麻省理工学院）
Given a set of images of a scene, the re-rendering of this scene from novel views and lighting conditions is an important and challenging problem in Computer Vision and Graphics. On the one hand, most existing works in Computer Vision usually impose many assumptions regarding the image formation process, e.g. direct illumination and predefined materials, to make scene parameter estimation tractable. On the other hand, mature Computer Graphics tools allow modeling of complex photo-realistic light transport given all the scene parameters. Combining these approaches, we propose a method for scene…
University of Tübingen（图宾根大学）；Google Research（谷歌研究院）
Inverse rendering of an object under entirely unknown capture conditions is a fundamental challenge in computer vision and graphics. Neural approaches such as NeRF have achieved photorealistic results on novel view synthesis, but they require known camera poses. Solving this problem with unknown camera poses is highly challenging as it requires joint optimization over shape, radiance, and pose. This problem is exacerbated when the input images are captured in the wild with varying backgrounds and illuminations. Standard pose estimation techniques fail in such image collections in the wild due …
GIR: 3D Gaussian Inverse Rendering for Relightable Scene Factorization
State Key Laboratory of Virtual Reality Technology and Systems, Beihang University（北京航空航天大学虚拟现实技术与系统国家重点实验室）；School of Electronic and Computer Engineering, Peking University（北京大学电子与计算机工程学院）；Baidu Inc.（百度公司）
University of Tübingen（图宾根大学）；Google Research（谷歌研究院）
Decomposing a scene into its shape, reflectance, and illumination is a challenging but important problem in computer vision and graphics. This problem is inherently more challenging when the illumination is not a single light source under laboratory conditions but is instead an unconstrained environmental illumination. Though recent work has shown that implicit representations can be used to model the radiance field of an object, most of these techniques only enable view synthesis and not relighting. Additionally, evaluating these radiance fields is resource and time-intensive. We propose a ne…
