S2D：基于稀疏到密集关键掩码蒸馏的无监督视频实例分割
S2D: Sparse-To-Dense Keymask Distillation for Unsupervised Video Instance Segmentation
AI总结：
S2D通过稀疏到密集关键掩码蒸馏方法，利用深度运动先验提升无监督视频实例分割的性能。
摘要：
近年来，无监督视频实例分割的最先进方法严重依赖于由对象中心图像数据集（如ImageNet）生成的合成视频数据。然而，通过人工位移和缩放图像实例掩码生成视频合成，无法准确建模视频中的真实运动，例如视角变化、单个或多个实例部分的运动或摄像机运动。为了解决这个问题，我们提出了一种仅在真实视频数据上训练的无监督视频实例分割模型。我们从单帧无监督实例分割掩码开始。然而，这些单帧分割存在时间噪声，其质量在视频中会变化。因此，我们通过利用深度运动先验来识别视频中的高质量关键掩码，以建立时间一致性。然后，稀疏关键掩码伪标注被用来训练一个分割模型进行隐式掩码传播，为此我们提出了一个由时间丢弃损失辅助的稀疏到密集蒸馏方法。在使用最终的密集标签集训练模型后，我们的方法在各种基准上均优于当前最先进的方法。
英文摘要：
In recent years, the state-of-the-art in unsupervised video instance segmentation has heavily relied on synthetic video data, generated from object-centric image datasets such as ImageNet. However, video synthesis by artificially shifting and scaling image instance masks fails to accurately model realistic motion in videos, such as perspective changes, movement by parts of one or multiple instances, or camera motion. To tackle this issue, we propose an unsupervised video instance segmentation model trained exclusively on real video data. We start from unsupervised instance segmentation masks on individual video frames. However, these single-frame segmentations exhibit temporal noise and their quality varies through the video. Therefore, we establish temporal coherence by identifying high-quality keymasks in the video by leveraging deep motion priors. The sparse keymask pseudo-annotations are then used to train a segmentation model for implicit mask propagation, for which we propose a Sparse-To-Dense Distillation approach aided by a Temporal DropLoss. After training the final model on the resulting dense labelset, our approach outperforms the current state-of-the-art across various benchmarks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
