arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2203.12344cs.CV
收藏

你是怎么做的?使用伪指针的细粒度动作理解

How Do You Do It? Fine-Grained Action Understanding with Pseudo-Adverbs

University of Amsterdam(阿姆斯特丹大学)

Hazel Doughty, Cees G. M. Snoek

摘要:

我们的目标是了解动作是如何执行的,并识别细微的差异,例如“坚定地折叠”与“温柔地折叠”。为此,我们提出了一种方法,认识到副词在不同的行动。然而,这样的细粒度的注释是很难获得的,他们的长尾性质使得它具有挑战性的识别副词在罕见的动作副词组成。因此,我们的方法使用具有多个副词伪标签的半监督学习来利用仅具有动作标签的视频。结合这些伪副词的自适应阈值,我们能够有效地利用现有的数据,同时解决长尾分布。此外,我们收集了三个现有的视频检索数据集的副词注释,这使我们能够引入识别看不见的动作副词组合和看不见的域中的副词的新任务。实验表明,我们的方法的有效性,它优于以前的工作,在识别副词和半监督的作品适用于副词识别。我们还展示了副词如何与细粒度的行动。

英文摘要:

We aim to understand how actions are performed and identify subtle differences, such as 'fold firmly' vs. 'fold gently'. To this end, we propose a method which recognizes adverbs across different actions. However, such fine-grained annotations are difficult to obtain and their long-tailed nature makes it challenging to recognize adverbs in rare action-adverb compositions. Our approach therefore uses semi-supervised learning with multiple adverb pseudo-labels to leverage videos with only action labels. Combined with adaptive thresholding of these pseudo-adverbs we are able to make efficient use of the available data while tackling the long-tailed distribution. Additionally, we gather adverb annotations for three existing video retrieval datasets, which allows us to introduce the new tasks of recognizing adverbs in unseen action-adverb compositions and unseen domains. Experiments demonstrate the effectiveness of our method, which outperforms prior work in recognizing adverbs and semi-supervised works adapted for adverb recognition. We also show how adverbs can relate fine-grained actions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑