ActionVOS：将动作作为视频对象分割的提示
ActionVOS: Actions as Prompts for Video Object Segmentation
AI总结：
针对现有指代视频对象分割依赖静态属性、难区分交互对象与状态变化对象的问题，提出以人类动作为提示的ActionVOS动作感知分割设定及配套方法，在多个数据集上验证了其分割性能提升效果。
摘要：
深入第一人称视觉领域，指代视频对象分割（RVOS）的发展对于理解人类活动至关重要。然而，现有RVOS任务主要依赖对象名称等静态属性来分割目标对象，这在区分目标对象与背景对象、识别状态发生变化的对象方面带来了挑战。为解决这些问题，本研究提出了一种全新的动作感知RVOS设定，名为ActionVOS，旨在利用人类动作作为关键语言提示，仅分割第一人称视频中的活跃对象。这是因为人类动作能够精准描述人类行为，从而帮助识别真正参与交互的对象，并理解可能发生的状态变化。我们还构建了一种适配该特定设定的方法。具体而言，我们开发了一个动作感知标注模块，并搭配高效的动作引导焦点损失（focal loss）。这些设计使ActionVOS模型能够借助现有的现成标注，优先处理活跃对象。在VISOR数据集上的实验结果表明，ActionVOS显著减少了对非活跃对象的错误分割，证实了动作能够帮助ActionVOS模型理解对象的参与状态。在VOST和VSCOS数据集上的进一步评估显示，全新的ActionVOS设定在遇到涉及对象状态变化的挑战性场景时，能够提升分割性能。我们将在https://github.com/ut-vision/ActionVOS上公开我们的实现代码。
英文摘要：
Delving into the realm of egocentric vision, the advancement of referring video object segmentation (RVOS) stands as pivotal in understanding human activities. However, existing RVOS task primarily relies on static attributes such as object names to segment target objects, posing challenges in distinguishing target objects from background objects and in identifying objects undergoing state changes. To address these problems, this work proposes a novel action-aware RVOS setting called ActionVOS, aiming at segmenting only active objects in egocentric videos using human actions as a key language prompt. This is because human actions precisely describe the behavior of humans, thereby helping to identify the objects truly involved in the interaction and to understand possible state changes. We also build a method tailored to work under this specific setting. Specifically, we develop an action-aware labeling module with an efficient action-guided focal loss. Such designs enable ActionVOS model to prioritize active objects with existing readily-available annotations. Experimental results on VISOR dataset reveal that ActionVOS significantly reduces the mis-segmentation of inactive objects, confirming that actions help the ActionVOS model understand objects' involvement. Further evaluations on VOST and VSCOS datasets show that the novel ActionVOS setting enhances segmentation performance when encountering challenging circumstances involving object state changes. We will make our implementation available at https://github.com/ut-vision/ActionVOS.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
