arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2211.01311cs.CV
收藏

半监督时态动作分割的提取与收集

Distill and Collect for Semi-Supervised Temporal Action Segmentation

Intel Labs(英特尔实验室)

Sovan Biswas, Anthony Rhodes, Ramesh Manuvinakurike, Giuseppe Raffa, Richard Beckwith

摘要:

最近的时间动作分割方法需要在训练期间进行帧注释才能有效。获得这些注释非常昂贵且耗时。当只有有限的注释数据可用时,这限制了它们的性能。相比之下,我们可以通过互联网轻松收集大量未注释的域内视频。因此,本文提出了一种方法的时间动作分割任务,可以同时利用知识的注释和未注释的视频序列。我们的方法使用多流蒸馏,反复精炼并最终组合它们的帧预测。我们的模型还预测了动作顺序,该顺序后来被用作时间约束,同时估计帧标签,以应对未注释视频缺乏监督的情况。最后,我们在两个不同的数据集上对所提出的方法进行的评估表明,尽管注释有限,但它能够实现与全面监督相当的性能。

英文摘要:

Recent temporal action segmentation approaches need frame annotations during training to be effective. These annotations are very expensive and time-consuming to obtain. This limits their performances when only limited annotated data is available. In contrast, we can easily collect a large corpus of in-domain unannotated videos by scavenging through the internet. Thus, this paper proposes an approach for the temporal action segmentation task that can simultaneously leverage knowledge from annotated and unannotated video sequences. Our approach uses multi-stream distillation that repeatedly refines and finally combines their frame predictions. Our model also predicts the action order, which is later used as a temporal constraint while estimating frames labels to counter the lack of supervision for unannotated videos. In the end, our evaluation of the proposed approach on two different datasets demonstrates its capability to achieve comparable performance to the full supervision despite limited annotation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑