无线边缘大规模场景重建的联邦3D高斯泼溅
Federated 3D Gaussian Splatting for Large-Scale Scene Reconstruction at Wireless Edge
- The Chinese University of Hong Kong (Shenzhen)（香港中文大学（深圳））
- Shenzhen Future Network of Intelligence Institute（深圳未来智能网络研究院）
- Guangdong Provincial Key Laboratory of Future Networks of Intelligence（广东省未来智能网络重点实验室）
- University of Liverpool（利物浦大学）
- University of Science and Technology of China（中国科学技术大学）
- Beijing Institute of Technology（北京理工大学）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
针对无线边缘资源受限下大规模3D-GS训练难题，提出资源高效联邦学习框架，通过设备端轻量化与模型恢复机制，加速收敛并保持高渲染质量。
AI中文摘要：
三维高斯泼溅（3D-GS）已成为大规模场景重建的一种有前景的技术，因其高渲染效率和保真度。然而，在无线边缘训练大规模3D-GS模型面临多种技术挑战，包括边缘设备上有限的通信、计算和图形处理单元（GPU）内存资源，跨局部模型的结构不一致问题阻碍了它们的有效聚合，以及与原始视觉内容和相机参数相关的隐私泄露风险。为解决这些挑战，本文提出了一种新颖的资源高效联邦学习框架，用于在严重资源约束下高效训练大规模场景的3D-GS模型。首先，我们提出了一种设备端模型轻量化机制，该机制自适应地选择和剪枝高斯点以平衡渲染质量和训练效率。在该机制中，我们定量评估每个设备上不同高斯点的重要性以促进剪枝，并使用一种新颖的重要性与延迟比率准则来确定在GPU内存和计算/通信延迟约束下剪枝的高斯点数量。此外，我们开发了一种3D-GS模型恢复机制，该机制在不访问私有相机参数的情况下恢复局部3D-GS模型之间的结构一致性，使其能够有效聚合为全局模型。最后，大量实验表明，与最先进的联邦3D-GS基线相比，我们的方法显著加速了收敛，保持了高渲染质量，并减少了训练延迟。
英文摘要：
Three-dimensional (3D) Gaussian splatting (3D-GS) has emerged as a promising technique for large-scale scene reconstruction due to its high rendering efficiency and fidelity. However, the training of large-scale 3D-GS models at wireless edge faces various technical challenges including the limited communication, computation, and graphics processing unit (GPU) memory resources at edge devices, the structural inconsistency issue across local models hindering their effective aggregation, as well as privacy leakage risks associated with raw visual content and camera parameters. To address these challenges, this paper proposes a novel resource-efficient federated learning framework for efficiently training 3D-GS models of large scenes under severe resource constraints. First, we propose an on-device model lightweighting mechanism that adaptively selects and prunes Gaussian points to balance the rendering quality and training efficiency. In this mechanism, we quantitatively evaluate the importance of different Gaussian points at each device to facilitate the pruning, and use a novel importance-to-latency ratio criterion to determine the number of pruned Gaussian points under GPU memory and computation/communication latency constraints. Furthermore, we develop a 3D-GS model recovery mechanism that restores structural consistency across local 3D-GS models without accessing private camera parameters, enabling their effective aggregation towards a global model. Finally, extensive experiments show that our approach significantly accelerates convergence, maintains high rendering quality, and reduces training latency compared to state-of-the-art federated 3D-GS baselines.
