We present an efficient neural 3D scene representation for novel-view synthesis (NVS) in large-scale, dynamic urban areas. Existing works are not well suited for applications like mixed-reality or closed-loop simulation due to their limited visual quality and non-interactive rendering speeds. Recently, rasterization-based approaches have achieved high-quality NVS at impressive speeds. However, these methods are limited to small-scale, homogeneous data, i.e. they cannot handle severe appearance and geometry variations due to weather, season, and lighting and do not scale to larger, dynamic areas with thousands of images. We propose 4DGF, a neural scene representation that scales to large-scale dynamic urban areas, handles heterogeneous input data, and substantially improves rendering speeds. We use 3D Gaussians as an efficient geometry scaffold while relying on neural fields as a compact and flexible appearance model. We integrate scene dynamics via a scene graph at global scale while modeling articulated motions on a local level via deformations. This decomposed approach enables flexible scene composition suitable for real-world applications. In experiments, we surpass the state-of-the-art by over 3 dB in PSNR and more than 200 times in rendering speed.
HGS-Mapping: Online Dense Mapping Using Hybrid Gaussian Representation in Urban Scenes
Fudan University(复旦大学);School of Future Technology, Harbin Institute of Technology(哈尔滨工业大学未来技术学院);Department of Computer Science and Engineering, Shanghai Jiao Tong University(上海交通大学计算机科学与工程系)
VAD-GS: Visibility-Aware Densification for 3D Gaussian Splatting in Dynamic Urban Scenes
Shanghai Research Institute for Intelligent Autonomous Systems, Tongji University(同济大学智能自主系统研究所);College of Electronic and Information Engineering, Tongji University(同济大学电子与信息工程学院);National Key Laboratory of Human-Machine Hybrid Augmented Intelligence, Xi’an Jiaotong University(西安交通大学人机混合增强智能国家实验室)
Semantic segmentation of longitudinal thermal images for identification of hot and cool spots in urban areas
Berkeley Education Alliance for Research in Singapore(新加坡伯克利教育研究联盟);Robert Bosch Centre for Cyber-physical Systems(罗伯特·博世信息物理系统研究中心);Indian Institute of Science(印度科学学院);University of California, Berkeley(加州大学伯克利分校);National University of Singapore(新加坡国立大学)
3D Gaussian Splatting for Fine-grained Urban Scene Reconstruction via Semantic-guided Density Control and Loss Design
School of Computer Science, China University of Geosciences (Wuhan)(中国地质大学(武汉)计算机学院);School of Architecture and Urban Planning, Shenzhen University(深圳大学建筑与城市规划学院);Guangdong Key Laboratory of Urban Informatics, Shenzhen University(深圳大学广东省城市信息学重点实验室);MNR Key Laboratory for Geo-Environmental Monitoring of Great Bay Area, Shenzhen University(深圳大学自然资源部大湾区地理环境监测重点实验室)