Neural rendering has demonstrated remarkable success in high-quality 3D neural reconstruction and novel view synthesis with dense input views and accurate poses. However, applying it to extremely sparse, unposed views in unbounded 360° scenes remains a challenging problem. In this paper, we propose a novel neural rendering framework to accomplish the unposed and extremely sparse-view 3D reconstruction in unbounded 360° scenes. To resolve the spatial ambiguity inherent in unbounded scenes with sparse input views, we propose a layered Gaussian-based representation to effectively model the scene with distinct spatial layers. By employing a dense stereo reconstruction model to recover coarse geometry, we introduce a layer-specific bootstrap optimization to refine the noise and fill occluded regions in the reconstruction. Furthermore, we propose an iterative fusion of reconstruction and generation alongside an uncertainty-aware training approach to facilitate mutual conditioning and enhancement between these two processes. Comprehensive experiments show that our approach outperforms existing state-of-the-art methods in terms of rendering quality and surface reconstruction accuracy. Project page: https://zju3dv.github.io/free360/
DynSUP: Dynamic Gaussian Splatting from An Unposed Image Pair
Technical University of Munich(慕尼黑工业大学);Munich Center for Machine Learning(慕尼黑机器学习中心);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州))
EvTrajGS: Accurate and Efficient 3D Gaussian Splatting from Unposed Event Streams
School of Computer Science and Engineering, Sun Yat-sen University(中山大学计算机科学与工程学院);Guangdong Province Key Laboratory of Information Security Technology(广东省信息安全技术重点实验室);Key Laboratory of Machine Intelligence and Advanced Computing, Ministry of Education(机器智能与先进计算教育部重点实验室);College of Computing and Data Science, Nanyang Technological University(南洋理工大学计算与数据科学学院);School of Computing and Information Technology, Great Bay University(大湾区大学计算与信息技术学院)
Pose-Free Omnidirectional Gaussian Splatting for 360-Degree Videos with Consistent Depth Priors
School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Chongqing University(重庆大学);Air Force Engineering University(空军工程大学)
GrapeSplat: Geometry-Grounded Reconstruction via Amalgamated Pose-Free Encoding for Feed-Forward 3D Gaussian Splatting
National Taiwan University(国立台湾大学);National Taiwan University of Science and Technology(国立台湾科技大学);NTU AI Center of Research Excellence (NTU AI-CoRE)(台湾大学人工智能卓越研究中心);VinUniversity(Vin大学)
TokenSplat: Token-aligned 3D Gaussian Splatting for Feed-forward Pose-free Reconstruction
State Key Laboratory of Complex and Critical Software Environment(复杂与关键软件环境国家重点实验室);School of Computer Science and Engineering(计算机科学与工程学院);School of Artificial Intelligence(人工智能学院)
GroupForward: Building Referable 3D Scenes via Instance-Grouped Feed-Forward Gaussian Splatting
School of Computer Science, Shanghai Jiao Tong University(上海交通大学计算机科学学院);School of Computer Science and Engineering, Beihang University(北京航空航天大学计算机科学与工程学院);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);School of Computer Science and Technology, East China Normal University(华东师范大学计算机科学与技术学院)
SaLon3R: Structure-aware Long-term Generalizable 3D Reconstruction from Unposed Images
The Chinese University of Hong Kong(香港中文大学);Hong Kong Center for Logistics Robotics(香港物流机器人中心);University of California, Berkeley(加州大学伯克利分校);Zhejiang University(浙江大学)
PLANA3R: Zero-shot Metric Planar 3D Reconstruction via Feed-Forward Planar Splatting
The Hong Kong University of Science and Technology(香港科技大学);Ant Group(蚂蚁集团);Wuhan University(武汉大学);Zhejiang University(浙江大学);The Pennsylvania State University(宾夕法尼亚州立大学)
GBR: Generative Bundle Refinement for High-fidelity Gaussian Splatting with Enhanced Mesh Reconstruction
College of future information technology, Fudan University(未来信息科技学院,复旦大学);School of Biomedical Engineering, Tsinghua University(生物医学工程学院,清华大学);Key Laboratory for Information Science of Electromagnetic Waves (MoE), Fudan University(电磁波信息科学重点实验室(MoE),复旦大学);Department of Automation, Tsinghua University(自动化系,清华大学);MoE Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University(人工智能重点实验室,人工智能研究院,上海交通大学)
ReCoSplat: Online Feed-Forward Gaussian Splatting via Render-and-Compare
University of California Merced(加州大学默塞德分校);ETH Zurich(苏黎世联邦理工学院);NVIDIA;Shanghai Jiao Tong University(上海交通大学);Hong Kong University of Science and Technology(香港科技大学)
Feat2GS: Probing Visual Foundation Models with Gaussian Splatting
Zhejiang University(浙江大学);Westlake University(西湖大学);Max Planck Institute for Informatics(马克斯·普朗克信息学研究所);Max Planck Institute for Intelligent Systems(马克斯·普朗克智能系统研究所)