arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2212.08816cs.CV
收藏

利用运动-外观协同改进无监督视频对象分割

Improving Unsupervised Video Object Segmentation with Motion-Appearance Synergy

Microsoft Research Asia(微软亚洲研究院);University of Michigan(密歇根大学)

Long Lian, Zhirong Wu, Stella X. Yu

摘要:

我们提出了IMAS,一种在训练或推理中无需手动注释即可分割视频中主要对象的方法。以前的方法在无监督视频对象分割(UVOS)已经证明了有效的运动作为输入或监督分割。然而,在诸如可变形对象和具有反射的对象的情况下,运动信号可能是无信息的甚至是误导的,从而导致不令人满意的分割。相比之下,IMAS通过运动-外观协同作用实现了改善的UVOS。我们的方法有两个训练阶段:1)运动监督的对象发现阶段,通过可学习的残差路径处理运动外观冲突; 2)具有低级和高级外观监督的细化阶段,以纠正从误导性运动线索中学习到的模型误解。此外,我们提出了运动语义对齐作为一个模型不可知的注释免费hyperparam调整方法。我们证明了它的有效性,在调整关键hyperparams以前调整与人类注释或手工制作的hyperparams特定的指标。IMAS大大提高了几个常见的UVOS基准分割质量。例如,在DAVIS 16基准测试中,我们仅使用标准ResNet和卷积头就超过了之前的方法8.3%。我们打算发布我们的代码用于未来的研究和应用。

英文摘要:

We present IMAS, a method that segments the primary objects in videos without manual annotation in training or inference. Previous methods in unsupervised video object segmentation (UVOS) have demonstrated the effectiveness of motion as either input or supervision for segmentation. However, motion signals may be uninformative or even misleading in cases such as deformable objects and objects with reflections, causing unsatisfactory segmentation. In contrast, IMAS achieves Improved UVOS with Motion-Appearance Synergy. Our method has two training stages: 1) a motion-supervised object discovery stage that deals with motion-appearance conflicts through a learnable residual pathway; 2) a refinement stage with both low- and high-level appearance supervision to correct model misconceptions learned from misleading motion cues. Additionally, we propose motion-semantic alignment as a model-agnostic annotation-free hyperparam tuning method. We demonstrate its effectiveness in tuning critical hyperparams previously tuned with human annotation or hand-crafted hyperparam-specific metrics. IMAS greatly improves the segmentation quality on several common UVOS benchmarks. For example, we surpass previous methods by 8.3% on DAVIS16 benchmark with only standard ResNet and convolutional heads. We intend to release our code for future research and applications.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑