利用张量核心加速3D高斯散射
Accelerating 3D Gaussian Splatting using Tensor Cores
AI总结:
本文提出TensorGS框架,通过将3D高斯散射的主导渲染计算转换为张量核心兼容的矩阵运算,并引入跨瓷砖分组以提高高斯重用率,从而在保持图像质量的同时提升端到端渲染性能1.65倍。
摘要:
3D高斯散射(3DGS)已成为实时神经渲染和3D场景重建的领先技术,但其渲染成本仍对许多延迟敏感的场景来说过高。特别是,3DGS中的光栅化阶段主导了端到端渲染时间,其中渲染器反复评估每个高斯对每个覆盖像素的贡献,使该阶段成为计算瓶颈。同时,现代GPU提供了高吞吐量的张量核心用于低精度矩阵运算,但现有的3DGS系统完全在CUDA核心上执行光栅化,使张量核心闲置。我们发现3DGS渲染可以在FP16中执行,质量损失可以忽略不计,表明了利用张量核心加速的有希望的机会。然而,利用张量核心进行3DGS并非易事,因为光栅化不自然匹配其执行模型。现有的3DGS光栅化以不规则的每像素标量运算表达,而张量核心需要密集、规则且重用丰富的矩阵工作负载。此外,传统逐瓷砖执行无法利用相邻瓷砖中的高斯重用,导致重复的数据加载,从而产生高数据移动开销。为此,我们提出了TensorGS,一个利用张量核心的3DGS加速框架。TensorGS将主导的光栅化计算转换为张量核心兼容的矩阵运算,并引入跨瓷砖分组以提高高斯重用率,摊销开销并提高张量核心利用率。实验结果表明,TensorGS在保持图像质量的同时,将端到端渲染性能提高了1.65倍。
英文摘要:
3D Gaussian Splatting (3DGS) has become a leading technique for real-time neural rendering and 3D scene reconstruction, but its rendering cost remains too high for many latency-sensitive scenarios. In particular, the rasterization stage in 3DGS dominates end-to-end rendering time, during which the renderer repeatedly evaluates each Gaussian's contribution to each covered pixel, making this stage compute-bound. At the same time, modern GPUs provide high-throughput Tensor Cores for low-precision matrix operations, yet existing 3DGS systems execute rasterization entirely on CUDA cores and leave Tensor Cores idle. We find that 3DGS rendering can be executed in FP16 with negligible quality degradation, suggesting a promising opportunity for Tensor Core acceleration. However, exploiting Tensor Cores for 3DGS is non-trivial because rasterization does not naturally match their execution model. Existing 3DGS rasterization is expressed as irregular per-pixel scalar operations, whereas Tensor Cores require dense, regular, and reuse-rich matrix workloads. Moreover, conventional tile-by-tile execution fails to exploit Gaussian reuse across neighboring tiles, resulting in repeated data loading and thus high data movement overhead. To this end, we present TensorGS, a 3DGS acceleration framework using Tensor Cores. TensorGS tensorizes the dominant rasterization computation into Tensor-Core-compatible matrix operations and introduces cross-tile grouping to improve Gaussian reuse, amortize overhead, and increase Tensor Core utilization. Experimental results show that TensorGS improves end-to-end rendering performance by 1.65$\times$ while preserving image quality.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。