利用运动信息以实现更好的自监督视频对应学习
Leveraging Motion Information for Better Self-Supervised Video Correspondence Learning
AI总结：
提出自监督视频对应学习框架MER，通过运动增强引擎和多簇采样策略捕捉动态运动信息，在视频对象分割和关键点跟踪任务上优于现有方法。
摘要：
自监督视频对应学习依赖于在对应同一视觉对象的视频帧之间准确关联像素的能力。然而，在没有监督的情况下实现可靠的像素匹配仍然是一个主要挑战。为了解决这个问题，最近的研究集中在旨在编码用于匹配的唯一像素表示的特征学习技术上。尽管取得了这些进展，现有方法仍然难以实现精确的像素对应，并且经常出现错误匹配，限制了它们在自监督环境中的有效性。为此，我们探索了一种高效的自监督视频对应学习框架（MER），旨在从无标签视频中准确提取对象细节。首先，我们设计了一个专用的运动增强引擎，强调捕捉视频中对象的动态运动。此外，我们引入了一种用于像素间对应信息的灵活采样策略，使模型能够更加关注运动中重要对象的像素变化。通过实验，我们的算法在视频对象分割和视频对象关键点跟踪等视频对应学习任务上优于最先进的竞争对手。
英文摘要：
Self-supervised video correspondence learning depends on the ability to accurately associate pixels between video frames that correspond to the same visual object. However, achieving reliable pixel matching without supervision remains a major challenge. To address this issue, recent research has focused on feature learning techniques that aim to encode unique pixel representations for matching. Despite these advances, existing methods still struggle to achieve exact pixel correspondences and often suffer from false matches, limiting their effectiveness in self-supervised settings. To this end, we explore an efficient self-supervised Video Correspondence Learning framework (MER) that aims to accurately extract object details from unlabeled videos. First, we design a dedicated Motion Enhancement Engine that emphasizes capturing the dynamic motion of objects in videos. In addition, we introduce a flexible sampling strategy for inter-pixel correspondence information (Multi-Cluster Sampler) that enables the model to pay more attention to the pixel changes of important objects in motion. Through experiments, our algorithm outperforms the state-of-the-art competitors on video correspondence learning tasks such as video object segmentation and video object keypoint tracking.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
