我们提出了 VicaSplat，这是一个新颖的框架，用于从一系列无位姿视频帧中联合进行 3D Gaussians 重建和相机位姿估计；在真实世界的 3D 应用中，这是一项关键但尚未得到充分研究的任务。我们方法的核心在于一种新颖的基于 transformer 的网络架构。具体而言，我们的模型首先使用图像编码器，将每张图像映射为一组视觉 token。所有视觉 token 都与额外插入的可学习相机 token 拼接。随后，所得 token 在一个定制的 transformer 解码器中彼此充分通信。相机 token 以因果方式聚合来自不同视图视觉 token 的特征，并进一步逐帧调制这些特征，以注入依赖于视图的特征。之后，可通过不同的预测头估计 3D Gaussian splats 和相机位姿参数。实验表明，VicaSplat 在多视图输入上优于基线方法，并取得了与此前双视图方法相当的性能。值得注意的是，VicaSplat 还在 ScanNet 基准上展示出出色的跨数据集泛化能力，在未经任何微调的情况下取得了优越性能。项目页面：https://lizhiqi49.github.io/VicaSplat。
英文摘要：
We present VicaSplat, a novel framework for joint 3D Gaussians reconstruction and camera pose estimation from a sequence of unposed video frames, which is a critical yet underexplored task in real-world 3D applications. The core of our method lies in a novel transformer-based network architecture. In particular, our model starts with an image encoder that maps each image to a list of visual tokens. All visual tokens are concatenated with additional inserted learnable camera tokens. The obtained tokens then fully communicate with each other within a tailored transformer decoder. The camera tokens causally aggregate features from visual tokens of different views, and further modulate them frame-wisely to inject view-dependent features. 3D Gaussian splats and camera pose parameters can then be estimated via different prediction heads. Experiments show that VicaSplat surpasses baseline methods for multi-view inputs, and achieves comparable performance to prior two-view approaches. Remarkably, VicaSplat also demonstrates exceptional cross-dataset generalization capability on the ScanNet benchmark, achieving superior performance without any fine-tuning. Project page: https://lizhiqi49.github.io/VicaSplat.
DynSUP: Dynamic Gaussian Splatting from An Unposed Image Pair
Technical University of Munich（慕尼黑工业大学）；Munich Center for Machine Learning（慕尼黑机器学习中心）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））
EvTrajGS: Accurate and Efficient 3D Gaussian Splatting from Unposed Event Streams
School of Computer Science and Engineering, Sun Yat-sen University（中山大学计算机科学与工程学院）；Guangdong Province Key Laboratory of Information Security Technology（广东省信息安全技术重点实验室）；Key Laboratory of Machine Intelligence and Advanced Computing, Ministry of Education（机器智能与先进计算教育部重点实验室）；College of Computing and Data Science, Nanyang Technological University（南洋理工大学计算与数据科学学院）；School of Computing and Information Technology, Great Bay University（大湾区大学计算与信息技术学院）
Pose-Free Omnidirectional Gaussian Splatting for 360-Degree Videos with Consistent Depth Priors
School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Chongqing University（重庆大学）；Air Force Engineering University（空军工程大学）
GrapeSplat: Geometry-Grounded Reconstruction via Amalgamated Pose-Free Encoding for Feed-Forward 3D Gaussian Splatting
National Taiwan University（国立台湾大学）；National Taiwan University of Science and Technology（国立台湾科技大学）；NTU AI Center of Research Excellence (NTU AI-CoRE)（台湾大学人工智能卓越研究中心）；VinUniversity（Vin大学）
TokenSplat: Token-aligned 3D Gaussian Splatting for Feed-forward Pose-free Reconstruction
State Key Laboratory of Complex and Critical Software Environment（复杂与关键软件环境国家重点实验室）；School of Computer Science and Engineering（计算机科学与工程学院）；School of Artificial Intelligence（人工智能学院）
GroupForward: Building Referable 3D Scenes via Instance-Grouped Feed-Forward Gaussian Splatting
School of Computer Science, Shanghai Jiao Tong University（上海交通大学计算机科学学院）；School of Computer Science and Engineering, Beihang University（北京航空航天大学计算机科学与工程学院）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；School of Computer Science and Technology, East China Normal University（华东师范大学计算机科学与技术学院）
SaLon3R: Structure-aware Long-term Generalizable 3D Reconstruction from Unposed Images
The Chinese University of Hong Kong（香港中文大学）；Hong Kong Center for Logistics Robotics（香港物流机器人中心）；University of California, Berkeley（加州大学伯克利分校）；Zhejiang University（浙江大学）
PLANA3R: Zero-shot Metric Planar 3D Reconstruction via Feed-Forward Planar Splatting
The Hong Kong University of Science and Technology（香港科技大学）；Ant Group（蚂蚁集团）；Wuhan University（武汉大学）；Zhejiang University（浙江大学）；The Pennsylvania State University（宾夕法尼亚州立大学）
GBR: Generative Bundle Refinement for High-fidelity Gaussian Splatting with Enhanced Mesh Reconstruction
College of future information technology, Fudan University（未来信息科技学院，复旦大学）；School of Biomedical Engineering, Tsinghua University（生物医学工程学院，清华大学）；Key Laboratory for Information Science of Electromagnetic Waves (MoE), Fudan University（电磁波信息科学重点实验室（MoE），复旦大学）；Department of Automation, Tsinghua University（自动化系，清华大学）；MoE Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University（人工智能重点实验室，人工智能研究院，上海交通大学）
ReCoSplat: Online Feed-Forward Gaussian Splatting via Render-and-Compare
University of California Merced（加州大学默塞德分校）；ETH Zurich（苏黎世联邦理工学院）；NVIDIA；Shanghai Jiao Tong University（上海交通大学）；Hong Kong University of Science and Technology（香港科技大学）
Feat2GS: Probing Visual Foundation Models with Gaussian Splatting
Zhejiang University（浙江大学）；Westlake University（西湖大学）；Max Planck Institute for Informatics（马克斯·普朗克信息学研究所）；Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）
Dental3R: Geometry-Aware Pairing for Intraoral 3D Reconstruction from Sparse-View Photographs
School of AI and Advanced Computing, Xi'an Jiaotong-Liverpool University, China（人工智能与先进计算学院，西安交通大学利物浦大学，中国）；School of Electrical Engineering, Electronics and Computer Science, University of Liverpool, United Kingdom（电子工程与计算机科学学院，利物浦大学，英国）；School of Advanced Technology, Xi'an Jiaotong-Liverpool University, China（先进技术学院，西安交通大学利物浦大学，中国）；School of Physical Sciences, University of Liverpool, Liverpool, United Kingdom（物理科学学院，利物浦大学，英国）；School of Mathematics and Physics, Xi'an Jiaotong-Liverpool University, China（数学与物理学院，西安交通大学利物浦大学，中国）；Institute of Artificial Intelligence Innovation, Zhejiang University of Technology, China（人工智能创新研究院，浙江工业大学，中国）
