三重协作视频阴影检测
Triple-cooperative Video Shadow Detection
摘要:
近年来,单幅图像中的阴影检测受到了极大的研究兴趣。然而,很少的工作已经探索了动态场景中的阴影检测。瓶颈是缺乏一个完善的数据集与高质量的视频阴影检测注释。在这项工作中,我们收集了一个新的视频阴影检测数据集,其中包含120个视频,11685帧,涵盖60个对象类别,不同的长度,和不同的运动/照明条件。所有帧都使用高质量像素级阴影遮罩进行注释。据我们所知,这是第一个面向学习的视频阴影检测数据集。此外,我们开发了一个新的基线模型,命名为三重合作视频阴影检测网络(TVSD-Net)。它以合作的方式利用三重并行网络来学习视频内和视频间级别的区别性表示。在该网络中,提出了一个双门控共同注意力模块来约束来自同一视频中相邻帧的特征,同时引入辅助相似性损失来挖掘不同视频之间的语义信息。最后,我们对ViSha进行了全面的研究,评估了12种最先进的模型(包括单个图像阴影检测器,视频对象分割和显着性检测方法)。实验表明,我们的模型优于SOTA的竞争对手。
英文摘要:
Shadow detection in a single image has received significant research interest in recent years. However, much fewer works have been explored in shadow detection over dynamic scenes. The bottleneck is the lack of a well-established dataset with high-quality annotations for video shadow detection. In this work, we collect a new video shadow detection dataset, which contains 120 videos with 11, 685 frames, covering 60 object categories, varying lengths, and different motion/lighting conditions. All the frames are annotated with a high-quality pixel-level shadow mask. To the best of our knowledge, this is the first learning-oriented dataset for video shadow detection. Furthermore, we develop a new baseline model, named triple-cooperative video shadow detection network (TVSD-Net). It utilizes triple parallel networks in a cooperative manner to learn discriminative representations at intra-video and inter-video levels. Within the network, a dual gated co-attention module is proposed to constrain features from neighboring frames in the same video, while an auxiliary similarity loss is introduced to mine semantic information between different videos. Finally, we conduct a comprehensive study on ViSha, evaluating 12 state-of-the-art models (including single image shadow detectors, video object segmentation, and saliency detection methods). Experiments demonstrate that our model outperforms SOTA competitors.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。