发表机构
Nankai University; Nankai International Advanced Research Institute (NKIARI); Alibaba Group（南开大学; 南开国际先进研究院; 阿里巴巴集团）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
提出ABC-Inter，利用贝塞尔控制点消除运动模糊并建模非均匀运动，通过精确光流估计模块提升帧插值性能，在多个基准上达到最先进水平。
AI 中文摘要
在帧插值任务中，训练集中的运动模糊性导致模型生成模糊的中间帧。此外，推理期间帧间均匀运动的假设进一步导致生成的中间帧不准确。为解决这些挑战，我们提出了一种基于贝塞尔控制点的精确运动估计算法ABC-Inter，用于高效帧插值。具体而言，ABC-Inter通过解耦两帧特征并映射到相应坐标来设计精确光流估计模块（AFM），以更好地估计两帧之间的光流。此外，ABC-Inter通过引入使用输入帧和中间真实帧（gt）计算的贝塞尔控制点，消除了训练集中的运动模糊性。这使得模型在训练过程中能够估计两帧之间的精确光流，从而解决推理期间生成中间帧的模糊问题。得益于两帧之间更精确的光流估计，我们可以引入额外的帧，并直接使用多个光流来计算贝塞尔控制点，以建模非均匀运动，而无需重新训练模型。同时，为实现仅使用两帧估计非线性运动，我们还引入了一个新的贝塞尔控制点估计模块，该模块通过在第二阶段对模型进行微调，实现两帧之间更好的运动估计。实验结果表明，我们的ABC-Inter在多个基准数据集上达到了最先进的性能，并展现出优异的视觉感知效果。
英文摘要
In frame interpolation tasks, motion ambiguity in the training set causes models to generate blurry intermediate frames. Moreover, the assumption of uniform motion between frames during inference further leads to inaccuracies in the generated intermediate frames. To tackle these challenges, we propose an Accurate motion estimation algorithm with Bézier Control point, ABC-Inter, for efficient frame Interpolation. Specifically, ABC-Inter designs an Accurate Flow estimation Module (AFM) by decoupling two-frame features and mapping to corresponding coordinates to better estimate the optical flow between the two frames. Furthermore, ABC-Inter eliminates motion ambiguity in the training set by introducing Bézier control points that are computed using the input frames and the intermediate ground-truth (gt) frames. This allows the model to estimate accurate optical flow between two frames during the training process, thereby solving the blurriness problem in the generated intermediate frames during inference. Benefiting from the more accurate flow estimation between two frames, we can introduce additional frames and directly use multiple flows to calculate Bézier control points for modeling non-uniform motion without retraining the model. Simultaneously, to realize the estimation of non-linear motion using only two frames, we also introduce a new Bézier control point estimation module which achieves better motion estimation between the two frames by performing fine-tuning on the model in the second stage. Experimental results demonstrate that our ABC-Inter achieves state-of-the-art performance on multiple benchmark datasets and exhibits excellent visual perception.
CommentsAccepted by IEEE TIP. Code: https://github.com/SHH-Han/ABC-Inter
