arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2106.02638cs.CV
收藏

将对象与Transformers关联用于视频对象分割

Associating Objects with Transformers for Video Object Segmentation

Zhejiang University(浙江大学);Baidu Research(百度研究院);Beijing Jiaotong University(北京交通大学)

Zongxin Yang, Yunchao Wei, Yi Yang

摘要:

本文研究了如何实现更好、更有效的嵌入学习,以解决具有挑战性的多对象场景下的半监督视频对象分割问题。现有的方法学习解码具有单个正对象的特征,因此必须在多对象场景下分别匹配和分割每个目标,消耗多次计算资源。为了解决这个问题,我们提出了一个关联对象与Transformers(AOT)的方法来匹配和解码多个对象统一。具体地说,AOT采用一种识别机制将多个目标关联到同一个高维嵌入空间中。因此,我们可以同时处理多个对象的匹配和分割解码处理单个对象一样有效。为了充分模拟多目标关联,设计了一个长短期Transformer来构造层次匹配和传播。我们在多目标和单目标基准上进行了大量的实验,以研究具有不同复杂度的AOT变体网络。特别是,我们的R50-AOT-L在三个受欢迎的基准上优于所有最先进的竞争对手,即,YouTube-VOS(84.1% J&F)、DAVIS 2017(84.9%)和DAVIS 2016(91.1%),同时保持了超过$3\times$的多对象运行速度。同时,我们的AOT-T可以在上述基准上保持实时多目标速度。基于AOT,我们在第三届大型VOS挑战赛中排名第一。

英文摘要:

This paper investigates how to realize better and more efficient embedding learning to tackle the semi-supervised video object segmentation under challenging multi-object scenarios. The state-of-the-art methods learn to decode features with a single positive object and thus have to match and segment each target separately under multi-object scenarios, consuming multiple times computing resources. To solve the problem, we propose an Associating Objects with Transformers (AOT) approach to match and decode multiple objects uniformly. In detail, AOT employs an identification mechanism to associate multiple targets into the same high-dimensional embedding space. Thus, we can simultaneously process multiple objects' matching and segmentation decoding as efficiently as processing a single object. For sufficiently modeling multi-object association, a Long Short-Term Transformer is designed for constructing hierarchical matching and propagation. We conduct extensive experiments on both multi-object and single-object benchmarks to examine AOT variant networks with different complexities. Particularly, our R50-AOT-L outperforms all the state-of-the-art competitors on three popular benchmarks, i.e., YouTube-VOS (84.1% J&F), DAVIS 2017 (84.9%), and DAVIS 2016 (91.1%), while keeping more than $3\times$ faster multi-object run-time. Meanwhile, our AOT-T can maintain real-time multi-object speed on the above benchmarks. Based on AOT, we ranked 1st in the 3rd Large-scale VOS Challenge.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑