FrameRS:一种由自监督视频帧重建器与关键帧选择器组成的视频帧压缩模型
FrameRS: A Video Frame Compression Model Composed by Self supervised Video Frame Reconstructor and Key Frame Selector
AI总结:
本文提出FrameRS视频帧压缩模型,结合自监督重建器FrameMAE与基于CNN的关键帧选择器,仅保留约30%关键帧即可高效压缩视频,在计算效率和准确率上优于传统关键帧提取算法。
摘要:
本文提出了帧重建模型FrameRS,该模型由自监督视频帧重建器和关键帧选择器组成。帧重建器FrameMAE通过将图像掩码自编码器(MAE)的原理适配到视频场景而开发。关键帧选择器Frame Selector基于CNN架构构建,以FrameMAE编码器的高层语义信息作为输入,能够以较低计算成本预测关键帧。结合我们定制设计的Frame Selector,FrameMAE可以通过保留视频片段中约30%的关键帧来有效压缩视频。在性能方面,我们的模型展现出计算高效性和具有竞争力的准确率,相比传统关键帧提取算法有显著提升。实现代码已在Github上公开。
英文摘要:
In this paper, we present frame reconstruction model: FrameRS. It consists self-supervised video frame reconstructor and key frame selector. The frame reconstructor, FrameMAE, is developed by adapting the principles of the Masked Autoencoder for Images (MAE) for video context. The key frame selector, Frame Selector, is built on CNN architecture. By taking the high-level semantic information from the encoder of FrameMAE as its input, it can predicted the key frames with low computation costs. Integrated with our bespoke Frame Selector, FrameMAE can effectively compress a video clip by retaining approximately 30% of its pivotal frames. Performance-wise, our model showcases computational efficiency and competitive accuracy, marking a notable improvement over traditional Key Frame Extract algorithms. The implementation is available on Github
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。