动态场景的紧凑显式4D表示

中文摘要 AI 辅助

紧凑的动态场景表示必须同时保留随时间变化的表面以及从新视角渲染它们所需的外观。我们提出了Sparc4D，一种前馈自编码器，将已知相机的单目视频编码为稀疏的4D场景状态。静态特征在片段中共享，而空间锚定的时间槽压缩时变特征。稀疏解码器生成2D高斯曲面元，同时存储的源像素通过几何重投影保留精细纹理。该状态包括一个完整源帧和每第四帧采样的动态区域像素，以及学习到的特征和稀疏占用。对于32帧的MultiCamVideo片段，在随机窗口上平均为0.95M个32位等效值，在first-32协议上为0.92M。在first-32上，Sparc4D达到21.70 dB，而MoVieS为20.40 dB。在随机放置的窗口上，它们的PSNR分数相当。在禁用存储纹理的情况下，时间槽将时变特征状态压缩了中位数4.0倍，并将平均状态从1.04M减少到0.42M个值，而目标视角重建质量基本不变。无需在真实数据上进行微调，Sparc4D即可迁移到DyCheck和Neu3D，其中存储纹理改善了LPIPS，同时略微降低了PSNR。

英文摘要

A compact dynamic-scene representation must retain both the surfaces seen over time and the appearance needed to render them from new viewpoints. We present Sparc4D, a feed-forward autoencoder that encodes a monocular video with known cameras into a sparse 4D scene state. Static features are shared across the clip, while spatially anchored temporal slots compress time-varying features. A sparse decoder produces 2D Gaussian surfels, while stored source pixels preserve fine texture through geometric re-projection. The state includes one full source frame and dynamic-region pixels sampled every fourth frame, alongside learned features and sparse occupancy. For a 32-frame MultiCamVideo clip, it averages 0.95M 32-bit-equivalent values on random windows and 0.92M on the first-32 protocol. On first-32, Sparc4D reaches 21.70\,dB, compared with 20.40\,dB for MoVieS. On randomly placed windows, their PSNR scores are comparable. With stored texture disabled, temporal slots compress the time-varying feature state by a median $4.0\times$ and reduce the mean state from 1.04M to 0.42M values, with essentially unchanged target-view reconstruction quality. Without fine-tuning on real data, Sparc4D transfers to DyCheck and Neu3D, where stored texture improves LPIPS while slightly reducing PSNR.

发表机构

• College of William & Mary（威廉与玛丽学院）
• SparcAI Inc.（SparcAI公司）

机构由 AI 辅助整理，请以论文原文为准。