学习细粒度特征用于像素级视频对应关系
Learning Fine-Grained Features for Pixel-wise Video Correspondences
AI总结：
本文提出结合标注合成视频与未标注真实视频，通过对抗学习和从粗到细框架学习细粒度像素特征，在准确性和效率上超越现有像素级视频对应方法。
摘要：
视频分析任务在很大程度上依赖于识别不同帧中对应于同一视觉目标的像素。为了解决这一问题，近期研究提倡特征学习方法，旨在学习具有区分性的表示来匹配像素，尤其是以自监督方式进行。然而，这些方法在处理微小甚至单像素视觉目标时存在困难。像素级视频对应关系传统上与光流相关，但光流方法会导致确定性对应关系，并且在真实世界视频上缺乏鲁棒性。我们针对建立像素级对应关系的特征学习问题展开研究。受光流以及自监督特征学习的启发，我们提出在一个整体框架中不仅使用带标注的合成视频，还使用未标注的真实世界视频来学习细粒度表示。我们采用对抗学习方案来增强所学特征的泛化能力。此外，我们设计了一个从粗到细的框架以追求高计算效率。我们在一系列基于对应关系的任务上的实验结果表明，所提出的方法在准确性和效率方面均优于最先进的竞争对手。
英文摘要：
Video analysis tasks rely heavily on identifying the pixels from different frames that correspond to the same visual target. To tackle this problem, recent studies have advocated feature learning methods that aim to learn distinctive representations to match the pixels, especially in a self-supervised fashion. Unfortunately, these methods have difficulties for tiny or even single-pixel visual targets. Pixel-wise video correspondences were traditionally related to optical flows, which however lead to deterministic correspondences and lack robustness on real-world videos. We address the problem of learning features for establishing pixel-wise correspondences. Motivated by optical flows as well as the self-supervised feature learning, we propose to use not only labeled synthetic videos but also unlabeled real-world videos for learning fine-grained representations in a holistic framework. We adopt an adversarial learning scheme to enhance the generalization ability of the learned features. Moreover, we design a coarse-to-fine framework to pursue high computational efficiency. Our experimental results on a series of correspondence-based tasks demonstrate that the proposed method outperforms state-of-the-art rivals in both accuracy and efficiency.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
