arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新，休息一下吧，周末愉快~~
arXiv 2508.11183cs.CV

基于前馈二维高斯溅射标记化的通用视频表示

Versatile Video Representation via Feed-Forward 2D Gaussian Splatting Tokenization

Zhenghao Chen, Zicong Chen, Lei Liu, Yiming Wu, Dong Xu

备注：
Accepted by ACM MM 2026, Rio de Janeiro, Brazil
阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

AI总结：

本研究提出基于前馈2DGS标记化的GVT框架，通过STGE与GSP策略实现通用视频表示，在四项视频任务的多数据集评估中，其重建、压缩性能达先进水平，动作识别提升，生成性能可与MAGVIT-v2媲美。

摘要：

近期依赖固定网格、逐块标记化的视频表示方法通常在灵活性上表现有限，均匀分配固定数量的标记往往会导致低信息区域的过度编码；在时间维度上，若不明确区分静态与动态内容，减少冗余仍具挑战性。本研究提出Gaussian Video Transformer（GVT），这是一个基于前馈二维高斯溅射（2DGS）标记化方案的通用视频表示框架。我们首先从视频片段中提取潜在的刚性特征，并通过所提出的时空高斯嵌入（STGE）机制以前馈方式生成的一组2D高斯来表示这些特征。这类2D高斯不仅在光栅化过程中通过向高（低）信息内容区域分配更高（低）的渲染权重来增强空间适应性，还通过避免逐视频优化来提升泛化能力。为增强时间通用性，我们引入高斯集划分（GSP）策略，将2D高斯划分为静态集与动态集，该策略明确建模跨不同时间步共享的静态内容以及每个时间步特有的动态内容，从而实现紧凑表示。我们在UCF101、Kinetics和DAVIS数据集上针对视频重建、视频动作识别、视频压缩、视频生成四项任务评估GVT，结果显示其具备最先进的重建与压缩性能，动作识别性能有所提升，且视频生成性能可与基线MAGVIT-v2相媲美。

英文摘要：

Recent video representation methods that rely on fixed-grid, patch-wise tokenization often exhibit limited versatility.Spatially, uniformly allocating a fixed number of tokens often leads to over-encoding in low-information regions. Temporally, reducing redundancy remains challenging without explicitly distinguishing between static and dynamic content. In this work, we introduce the Gaussian Video Transformer (GVT), a versatile video representation framework built on a feed-forward 2D Gaussian Splatting (2DGS) tokenization scheme. We first extract latent rigid features from a video clip and represent them with a set of 2D Gaussians generated by our proposed Spatio-Temporal Gaussian Embedding (STGE) mechanism in a feed-forward manner. Such 2D Gaussians not only enhance spatial adaptability by assigning higher (resp., lower) rendering weights to regions with higher (resp., lower) information content during rasterization, but also improve generalization by avoiding per-video optimization. To enhance the temporal versatility, we introduce a Gaussian Set Partitioning (GSP) strategy that separates the 2D Gaussians into static and dynamic sets, which explicitly model static content shared across different time-steps and dynamic content specific to each time-step, enabling a compact representation. We evaluate GVT across four tasks: video reconstruction, video action recognition, video compression, and video generation, on the UCF101, Kinetics, and DAVIS datasets. The results demonstrate state-of-the-art reconstruction and compression performance, improved action recognition, and video generation performance comparable to the baseline MAGVIT-v2.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑