arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2501.02427cs.CV

MetaNeRV：具备时空引导的视频元神经表示

MetaNeRV: Meta Neural Representations for Videos with Spatial-Temporal Guidance

College of Computer Science, Zhejiang University（浙江大学计算机科学学院）；Cooperative Medianet Innovation Center, Shanghai Jiaotong University（上海交通大学合作媒体创新中心）

Jialong Guo, Ke liu, Jiangchao Yao, Zhihua Wang, Jiajun Bu, Haishuai Wang

备注：
Accepted by AAAI2025
阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对NeRV适配多视频效率低的问题，提出带时空引导的MetaNeRV框架，通过元学习获最优初始化，结合多分辨率空间损失与渐进时间学习策略，提升视频表示与压缩性能。

摘要：

视频神经表示（NeRV）已成为一种颇具前景的视频分析隐式神经表示（INR）方法，它将视频表示为以帧索引为输入的神经网络。然而，基于NeRV的方法在适配大量不同视频时非常耗时，因为每个视频都需要从头开始训练一个独立的NeRV模型。此外，基于NeRV的方法在空间上需要从低维度时间戳的输入生成高维度信号（即整幅图像），而视频在时间上通常由数十帧组成，相邻帧之间变化很小。为提升视频表示效率，我们提出视频元神经表示方法，命名为MetaNeRV，这是一种用于快速对未见过的视频进行NeRV表示的新型框架。MetaNeRV利用元学习框架学习最优参数初始化，该初始化可作为适配新视频的良好起点。为解决视频模态独特的时空特性问题，我们进一步引入时空引导以提升MetaNeRV的表示能力。具体而言，带有多分辨率损失的空间引导旨在捕获来自不同分辨率阶段的信息，而带有有效渐进学习策略的时间引导能够在元学习过程中逐步优化拟合帧的数量。在多个数据集上开展的大量实验证明了MetaNeRV在视频表示和视频压缩方面的优越性。

英文摘要：

Neural Representations for Videos (NeRV) has emerged as a promising implicit neural representation (INR) approach for video analysis, which represents videos as neural networks with frame indexes as inputs. However, NeRV-based methods are time-consuming when adapting to a large number of diverse videos, as each video requires a separate NeRV model to be trained from scratch. In addition, NeRV-based methods spatially require generating a high-dimension signal (i.e., an entire image) from the input of a low-dimension timestamp, and a video typically consists of tens of frames temporally that have a minor change between adjacent frames. To improve the efficiency of video representation, we propose Meta Neural Representations for Videos, named MetaNeRV, a novel framework for fast NeRV representation for unseen videos. MetaNeRV leverages a meta-learning framework to learn an optimal parameter initialization, which serves as a good starting point for adapting to new videos. To address the unique spatial and temporal characteristics of video modality, we further introduce spatial-temporal guidance to improve the representation capabilities of MetaNeRV. Specifically, the spatial guidance with a multi-resolution loss aims to capture the information from different resolution stages, and the temporal guidance with an effective progressive learning strategy could gradually refine the number of fitted frames during the meta-learning process. Extensive experiments conducted on multiple datasets demonstrate the superiority of MetaNeRV for video representations and video compression.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑