基于时空特征学习的卷积神经网络帧速率上转换检测
Frame-rate Up-conversion Detection Based on Convolutional Neural Network for Learning Spatiotemporal Features
摘要：
随着用户友好和强大的视频编辑工具的进步，任何人都可以轻松地操纵视频，而不会留下明显的视觉痕迹。帧速率上转换（FRUC）是一种代表性的时域操作，其增加了具有较低帧率的视频的运动连续性，并且被恶意伪造者用于视频篡改，例如生成假帧率视频而不提高质量或混合时间拼接的视频。FRUC基于帧插值方案，并且保留在插值帧中的细微伪影通常难以区分。因此，检测这种伪造痕迹是视频取证中的一个关键问题。本文提出了一种帧速率转换检测网络（FCDNet），它以端到端的方式学习FRUC引起的取证特征。所提出的网络使用连续帧的堆栈作为输入，并使用网络块来学习时空特征，从而有效地学习插值伪影。本研究首次尝试将神经网络应用于FRUC的检测。此外，它可以涵盖以下三种类型的帧内插方案：最近邻内插，双线性内插和运动补偿内插。与利用所有帧来验证完整性的现有方法相比，所提出的方法实现了高检测速度，因为它只观察六帧来测试其真实性。为了验证我们的研究成果，我们使用传统的取证方法和神经网络进行了大量的视频取证实验。该网络在检测FRUC的内插伪影方面达到了最先进的性能。实验结果还表明，我们的训练模型对于看不见的数据集，未学习的帧速率和未学习的质量因子是鲁棒的。
英文摘要：
With the advance in user-friendly and powerful video editing tools, anyone can easily manipulate videos without leaving prominent visual traces. Frame-rate up-conversion (FRUC), a representative temporal-domain operation, increases the motion continuity of videos with a lower frame-rate and is used by malicious counterfeiters in video tampering such as generating fake frame-rate video without improving the quality or mixing temporally spliced videos. FRUC is based on frame interpolation schemes and subtle artifacts that remain in interpolated frames are often difficult to distinguish. Hence, detecting such forgery traces is a critical issue in video forensics. This paper proposes a frame-rate conversion detection network (FCDNet) that learns forensic features caused by FRUC in an end-to-end fashion. The proposed network uses a stack of consecutive frames as the input and effectively learns interpolation artifacts using network blocks to learn spatiotemporal features. This study is the first attempt to apply a neural network to the detection of FRUC. Moreover, it can cover the following three types of frame interpolation schemes: nearest neighbor interpolation, bilinear interpolation, and motion-compensated interpolation. In contrast to existing methods that exploit all frames to verify integrity, the proposed approach achieves a high detection speed because it observes only six frames to test its authenticity. Extensive experiments were conducted with conventional forensic methods and neural networks for video forensic tasks to validate our research. The proposed network achieved state-of-the-art performance in terms of detecting the interpolated artifacts of FRUC. The experimental results also demonstrate that our trained model is robust for an unseen dataset, unlearned frame-rate, and unlearned quality factor.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
