微动作识别基准测试：数据集、方法与应用
Benchmarking Micro-action Recognition: Dataset, Methods, and Applications
AI总结：
本文构建了包含52类微动作的数据集MA-52，并提出融合SE和TSM的MANet基准，通过联合嵌入损失优化识别，验证了其在情绪识别等领域的应用价值。
摘要：
微动作是一种以低强度运动为特征的难以察觉的非语言行为。它为了解个人的感受和意图提供了深刻见解，对于情绪识别和心理评估等以人为导向的应用至关重要。然而，由于这些微妙的人类行为在日常生活中具有难以察觉和不可触及的特性，微动作的识别、区分和理解面临着巨大挑战。在本研究中，我们创新性地收集了一个新的微动作数据集，命名为 Micro-action-52 (MA-52)，并提出了一个名为 micro-action network (MANet) 的基准，用于微动作识别 (MAR) 任务。独特的是，MA-52 提供了包括手势、上肢和下肢运动在内的全身视角，试图揭示全面的微动作线索。具体而言，MA-52 包含 52 个微动作类别以及七个身体部位标签，涵盖了各种真实自然的微动作，涉及从心理访谈中整理的 205 名参与者和 22,422 个视频实例。基于所提出的数据集，我们评估了 MANet 及其他九种主流的动作识别方法。MANet 将 squeeze-and-excitation (SE) 和 temporal shift module (TSM) 引入 ResNet 架构中，以建模微动作的时空特征。随后，设计了一种联合嵌入损失用于视频和动作标签之间的语义匹配；该损失旨在更好地区分视觉上相似但截然不同的微动作类别。在情绪识别中的扩展应用证明了我们提出的数据集和方法的重要价值之一。未来，将进一步深入探索人类行为、情绪和心理评估。数据集和源代码已发布在 https://github.com/VUT-HFUT/Micro-Action。
英文摘要：
Micro-action is an imperceptible non-verbal behaviour characterised by low-intensity movement. It offers insights into the feelings and intentions of individuals and is important for human-oriented applications such as emotion recognition and psychological assessment. However, the identification, differentiation, and understanding of micro-actions pose challenges due to the imperceptible and inaccessible nature of these subtle human behaviors in everyday life. In this study, we innovatively collect a new micro-action dataset designated as Micro-action-52 (MA-52), and propose a benchmark named micro-action network (MANet) for micro-action recognition (MAR) task. Uniquely, MA-52 provides the whole-body perspective including gestures, upper- and lower-limb movements, attempting to reveal comprehensive micro-action cues. In detail, MA-52 contains 52 micro-action categories along with seven body part labels, and encompasses a full array of realistic and natural micro-actions, accounting for 205 participants and 22,422 video instances collated from the psychological interviews. Based on the proposed dataset, we assess MANet and other nine prevalent action recognition methods. MANet incorporates squeeze-and excitation (SE) and temporal shift module (TSM) into the ResNet architecture for modeling the spatiotemporal characteristics of micro-actions. Then a joint-embedding loss is designed for semantic matching between video and action labels; the loss is used to better distinguish between visually similar yet distinct micro-action categories. The extended application in emotion recognition has demonstrated one of the important values of our proposed dataset and method. In the future, further exploration of human behaviour, emotion, and psychological assessment will be conducted in depth. The dataset and source code are released at https://github.com/VUT-HFUT/Micro-Action.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
