基于密集查询双边相关性的视频帧插值
Video Frame Interpolation with Densely Queried Bilateral Correlation
AI总结：
针对视频帧插值中运动场估计的感受野依赖问题，提出 DQBC 密集查询双边相关性，并结合上下文细化与 SynthNet 合成中间帧，在精度和推理速度上优于现有方法。
摘要：
视频帧插值（Video Frame Interpolation，VFI）旨在合成已有帧之间不存在的中间帧。基于光流的 VFI 算法通过估计中间运动场来扭曲已有帧。真实世界运动的复杂性以及参考帧的缺失使运动估计颇具挑战。许多最先进方法显式建模两个相邻帧之间的相关性，以实现更准确的运动估计。在常见方法中，较高分辨率下相关性建模的感受野依赖于事先估计的运动场。这种感受野依赖性使常见运动估计方法难以应对小型且快速移动的物体。为更好地建模相关性并生成更准确的运动场，我们提出密集查询双边相关性（Densely Queried Bilateral Correlation，DQBC），它摆脱了感受野依赖问题，因此更适合小型且快速移动的物体。在 DQBC 帮助下生成的运动场会进一步利用上下文特征进行细化和上采样。运动场确定后，基于 CNN 的 SynthNet 合成最终的插值帧。实验表明，与最先进方法相比，我们的方法具有更高的精度和更少的推理时间。源代码可在 https://github.com/kinoud/DQBC 获取。
英文摘要：
Video Frame Interpolation (VFI) aims to synthesize non-existent intermediate frames between existent frames. Flow-based VFI algorithms estimate intermediate motion fields to warp the existent frames. Real-world motions' complexity and the reference frame's absence make motion estimation challenging. Many state-of-the-art approaches explicitly model the correlations between two neighboring frames for more accurate motion estimation. In common approaches, the receptive field of correlation modeling at higher resolution depends on the motion fields estimated beforehand. Such receptive field dependency makes common motion estimation approaches poor at coping with small and fast-moving objects. To better model correlations and to produce more accurate motion fields, we propose the Densely Queried Bilateral Correlation (DQBC) that gets rid of the receptive field dependency problem and thus is more friendly to small and fast-moving objects. The motion fields generated with the help of DQBC are further refined and up-sampled with context features. After the motion fields are fixed, a CNN-based SynthNet synthesizes the final interpolated frame. Experiments show that our approach enjoys higher accuracy and less inference time than the state-of-the-art. Source code is available at https://github.com/kinoud/DQBC.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
