arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2412.03061cs.CV
收藏

通过混合变形实现轻量级随机视频预测

Lightweight Stochastic Video Prediction via Hybrid Warping

Waseda University(早稻田大学);Yokohama National University(横滨国立大学)

Kazuki Kotoyori, Shota Hirose, Heming Sun, Jiro Katto

AI总结:

针对长期视频预测中运动动态、遮挡及不确定性问题,本文提出 SVPHW,融合前向与后向变形并采用基于 MobileNet 的轻量架构,在两个基准上达到最优性能。

摘要:

深度神经网络的准确视频预测,尤其是针对动态区域的预测,是计算机视觉中的一项挑战性任务,适用于自动驾驶、远程办公和远程医疗等关键应用。由于固有的不确定性,现有预测模型往往难以处理运动动态和遮挡的复杂性。本文提出一种新颖的随机长期视频预测模型,通过采用混合变形策略聚焦于动态区域。通过整合前向变形和后向变形生成的帧,该方法有效弥补了各项技术的弱点,提高了视频中运动区域的预测准确性和真实感,同时通过作出考虑各种运动的随机预测来处理不确定性。此外,考虑到实时预测,我们在模型中引入了基于 MobileNet 的轻量级架构。我们的模型称为 SVPHW,在两个基准数据集上取得了最先进的性能。

英文摘要:

Accurate video prediction by deep neural networks, especially for dynamic regions, is a challenging task in computer vision for critical applications such as autonomous driving, remote working, and telemedicine. Due to inherent uncertainties, existing prediction models often struggle with the complexity of motion dynamics and occlusions. In this paper, we propose a novel stochastic long-term video prediction model that focuses on dynamic regions by employing a hybrid warping strategy. By integrating frames generated through forward and backward warpings, our approach effectively compensates for the weaknesses of each technique, improving the prediction accuracy and realism of moving regions in videos while also addressing uncertainty by making stochastic predictions that account for various motions. Furthermore, considering real-time predictions, we introduce a MobileNet-based lightweight architecture into our model. Our model, called SVPHW, achieves state-of-the-art performance on two benchmark datasets.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑