S²GS：面向边缘物联网设备的高效自由视点视频重建的结构化稀疏高斯流
S$^2$GS: Structured Sparse Gaussian Streaming for Efficient Free-Viewpoint Video Reconstruction on Edge-IoT Devices
- The Hong Kong Polytechnic University（香港理工大学）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
S²GS是面向边缘物联网设备的FVV重建框架，通过结构化稀疏高斯流技术，在保持视觉质量的同时，大幅降低每帧优化时间和存储占用，在Jetson AGX Orin等设备上实现高吞吐量与低能耗。
AI中文摘要：
自由视点视频（FVV）的流重建支持沉浸式物联网（IoT）服务，如远程呈现和数字孪生可视化。现有方法存在每帧优化时间长、存储占用大的问题，限制了在资源受限的边缘物联网设备上的部署。为解决这些挑战，我们提出Structured Sparse Gaussian Streaming（S²GS，结构化稀疏高斯流），这是一种FVV重建框架，利用感知结构的时间稀疏性选择性更新高斯残差，在不损害视觉保真度的情况下实现高效流处理。在空间域，流八叉树分层组织高斯残差，捕捉指导残差更新的空间相关性；在时间域，结构化门控机制（包含分层特征传播HFP和Gumbel-Sigmoid采样）在可微优化下将分层动态线索转化为稀疏残差更新决策。进一步采用多级离散方案，在保留复杂动态细节的同时对残差更新进行细粒度控制。在消费级GPU、工业边缘物联网设备和物理远程呈现测试平台上开展的大量实验表明，S²GS在保持竞争力的视觉质量的同时，持续降低每帧优化时间和存储占用。与QUEEN相比，S²GS在RTX 4090 GPU上的每帧优化时间减少59%，存储成本降低85%；在Jetson AGX Orin上，S²GS在评估方法中实现最高渲染吞吐量（60+ FPS）和最低能耗，展现出在资源受限系统中部署的潜力。
英文摘要：
Streaming reconstruction of Free-Viewpoint Videos (FVVs) supports immersive Internet of Things (IoT) services, such as telepresence and digital twin visualization. Existing methods suffer from high per-frame optimization time and large storage footprints, limiting deployment on resource-constrained Edge-IoT devices. To address these challenges, we propose Structured Sparse Gaussian Streaming (S$^2$GS), an FVV reconstruction framework that exploits structure-aware temporal sparsity to selectively update Gaussian residuals, enabling efficient streaming without compromising visual fidelity. In the spatial domain, a streaming octree hierarchically organizes Gaussian residuals, capturing spatial correlations that guide residual updates. In the temporal domain, a structured gating mechanism, comprising hierarchical feature propagation (HFP) and Gumbel-Sigmoid sampling, converts hierarchical dynamic cues into sparse residual update decisions under differentiable optimization. A multi-level discrete scheme is further adopted to provide fine-grained control over residual updates while preserving intricate dynamic details. Extensive experiments across consumer GPUs, industrial edge IoT devices, and a physical telepresence testbed demonstrate that S$^2$GS consistently reduces per-frame optimization time and storage footprint while maintaining competitive visual quality. Compared with QUEEN, S$^2$GS reduces per-frame optimization time by 59% and storage costs by 85% on an RTX 4090 GPU. On the Jetson AGX Orin, S$^2$GS delivers the highest rendering throughput (60+ FPS) and the lowest energy consumption among the evaluated methods, demonstrating its potential for deployment in resource-constrained systems.
