TIVE：一种识别视频实例分割错误的方法
TIVE: A Toolbox for Identifying Video Instance Segmentation Errors
摘要：
视频实例分割（VIS）任务自提出以来，吸引了众多研究者对视频结构建模的关注。虽然在线和离线模式取得了很大的进步，但仍然没有足够的手段来识别模型错误并区分方法之间的差异，并且正确反映模型在识别各种时间长度的对象实例时的性能的方法仍然很少。更重要的是，作为任务所需的基本模型能力，空间分割和时间关联在评估和交互机制中仍然研究不足。在本文中，我们介绍了TIVE，一个用于识别视频实例分割错误的工具。通过直接操作输出预测文件，TIVE定义了孤立的错误类型，并对每种类型对mAP的损害进行加权，以区分模型特征。通过在时空维度上对定位质量进行分解，揭示了模型在空间分割和时间关联方面的潜在缺陷。TIVE还可以报告实际应用程序的实例时间长度上的mAP。我们通过工具箱进行了大量的实验，以进一步说明空间分割和时间关联如何相互影响。我们期望通过对TIVE的分析，能够给研究者提供更多的启示，引导社会各界对视频实例分割进行更有意义的探索。拟议的工具箱可在https://github.com/wenhe-jia/TIVE上查阅。
英文摘要：
Since first proposed, Video Instance Segmentation(VIS) task has attracted vast researchers' focus on architecture modeling to boost performance. Though great advances achieved in online and offline paradigms, there are still insufficient means to identify model errors and distinguish discrepancies between methods, as well approaches that correctly reflect models' performance in recognizing object instances of various temporal lengths remain barely available. More importantly, as the fundamental model abilities demanded by the task, spatial segmentation and temporal association are still understudied in both evaluation and interaction mechanisms. In this paper, we introduce TIVE, a Toolbox for Identifying Video instance segmentation Errors. By directly operating output prediction files, TIVE defines isolated error types and weights each type's damage to mAP, for the purpose of distinguishing model characters. By decomposing localization quality in spatial-temporal dimensions, model's potential drawbacks on spatial segmentation and temporal association can be revealed. TIVE can also report mAP over instance temporal length for real applications. We conduct extensive experiments by the toolbox to further illustrate how spatial segmentation and temporal association affect each other. We expect the analysis of TIVE can give the researchers more insights, guiding the community to promote more meaningful explorations for video instance segmentation. The proposed toolbox is available at https://github.com/wenhe-jia/TIVE.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
