Prototype Learning for Micro-gesture Classification
Hefei University of Technology (HFUT)（合肥工业大学）；Hefei Comprehensive National Science Center（合肥综合性国家科学中心）；Zhejiang University（浙江大学）；Anhui Zhonghuitong Technology Co., Ltd.（安徽中汇通科技有限公司）
Guoliang Chen, Fei Wang, Kun Li, Zhiliang Wu, Hehe Fan, Yi Yang, Meng Wang, Dan Guo
备注：
1st Place in Micro-gesture Classification in MiGA at IJCAI-2024
In this paper, we briefly introduce the solution developed by our team, HFUT-VUT, for the track of Micro-gesture Classification in the MiGA challenge at IJCAI 2024. The task of micro-gesture classification task involves recognizing the category of a given video clip, which focuses on more fine-grained and subtle body movements compared to typical action recognition tasks. Given the inherent complexity of micro-gesture recognition, which includes large intra-class variability and minimal inter-class differences, we utilize two innovative modules, i.e., the cross-modal fusion module and prototypical refinement module, to improve the discriminative ability of MG features, thereby improving the classification accuracy. Our solution achieved significant success, ranking 1st in the track of Micro-gesture Classification. We surpassed the performance of last year's leading team by a substantial margin, improving Top-1 accuracy by 6.13%.
Identity-free Artificial Emotional Intelligence via Micro-Gesture Understanding
Lappeenranta-Lahti University of Technology LUT（拉彭兰塔-拉赫蒂理工大学（LUT））；Great Bay University（大湾区大学）；Imperial College London（伦敦帝国理工学院）；Technical University of Munich（慕尼黑工业大学）
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）
Hand gesture recognition plays a significant role in human-computer interaction for understanding various human gestures and their intent. However, most prior works can only recognize gestures of limited labeled classes and fail to adapt to new categories. The task of Generalized Zero-Shot Learning (GZSL) for hand gesture recognition aims to address the above issue by leveraging semantic representations and detecting both seen and unseen class samples. In this paper, we propose an end-to-end prototype-based GZSL framework for hand gesture recognition which consists of two branches. The first b…
Data-Free Class-Incremental Gesture Recognition with Prototype-Guided Pseudo Feature Replay
School of Computer Science and Engineering, Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications, Ministry of Education, Southeast University（东南大学计算机科学与工程学院、教育部新一代人工智能技术及其跨学科应用重点实验室）；School of Cyber Science and Engineering, Engineering Research Center of Blockchain Application, Supervision And Management (Southeast University), Ministry of Education, Southeast University（东南大学网络空间安全学院、教育部区块链应用监管工程研究中心（东南大学））；New Laboratory of Pattern Recognition (NLPR), State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS), Institute of Automation, Chinese Academy of Sciences (CASIA)（中国科学院自动化研究所模式识别国家重点实验室、多模态人工智能系统全国重点实验室）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）
GMoT: Gated Motion-Aware Tokenization for Fine-Grained Micro-Gesture Video Reasoning with Multimodal LLMs
Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳））；Great Bay University（大湾区大学）；Dongguan Key Laboratory for Intelligence and Information Technology（东莞市智能信息技术重点实验室）
We introduce a new dataset for the emotional artificial intelligence research: identity-free video dataset for Micro-Gesture Understanding and Emotion analysis (iMiGUE). Different from existing public datasets, iMiGUE focuses on nonverbal body gestures without using any identity information, while the predominant researches of emotion analysis concern sensitive biometric data, like face and speech. Most importantly, iMiGUE focuses on micro-gestures, i.e., unintentional behaviors driven by inner feelings, which are different from ordinary scope of gestures from other gesture datasets which are …
Micro-gesture Online Recognition using Learnable Query Points
School of Computer Science and Information Engineering（计算机与信息工程学院）；School of Artificial Intelligence（人工智能学院）；Hefei University of Technology (HFUT)（合肥工业大学）；Key Laboratory of Knowledge Engineering with Big Data (HFUT), Ministry of Education（大数据知识工程教育部重点实验室）；Institute of Artificial Intelligence, Hefei Comprehensive National Science Center（合肥综合性国家科学中心人工智能研究院）；Zhejiang University（浙江大学）；Anhui Zhonghuitong Technology Co., Ltd.（安徽中汇通科技有限公司）
TU Munich（慕尼黑工业大学）；LMU Munich（慕尼黑大学）；University of Tuebingen（图宾根大学）；Intel Labs Europe（英特尔欧洲实验室）；Intel Deutschland GmbH（英特尔德国有限公司）
The use of hand gestures provides a natural alternative to cumbersome interface devices for Human-Computer Interaction (HCI) systems. However, real-time recognition of dynamic micro hand gestures from video streams is challenging for in-vehicle scenarios since (i) the gestures should be performed naturally without distracting the driver, (ii) micro hand gestures occur within very short time intervals at spatially constrained areas, (iii) the performed gesture should be recognized only once, and (iv) the entire architecture should be designed lightweight as it will be deployed to an embedded sy…
Online Micro-gesture Recognition Using Data Augmentation and Spatial-Temporal Attention
School of Computer Science and Information Engineering, School of Artificial Intelligence, Hefei University of Technology (HFUT)（计算机科学与信息工程学院、人工智能学院，合肥工业大学）；ReLER, CCAI, Zhejiang University, China（ReLER、CCAI、浙江大学，中国）；Institute of Advanced Technology, University of Science and Technology of China（先进技术研究院，中国科学技术大学）；Institute of Artificial Intelligence, Hefei Comprehensive National Science Center, China（人工智能研究院，合肥综合性国家科学中心，中国）；Key Laboratory of Knowledge Engineering with Big Data (HFUT), Ministry of Education（大数据知识工程重点实验室（HFUT），教育部）；Xinsight Lab, Research Institute, Hefei Zhongjuyuan Intelligent Technology Co., Ltd., China（Xinsight实验室、研究院，合肥中 jouyuan智能科技有限公司，中国）
MM-Gesture: Towards Precise Micro-Gesture Recognition through Multimodal Fusion
University College London (UCL)（伦敦大学学院）；School of Computer Science（计算机科学学院）；Information Engineering, School of Artificial Intelligence, Hefei University of Technology (HFUT)（信息工程学院，人工智能学院，合肥工业大学）；ReLER, CCAI, Zhejiang University, China（ReLER、CCAI、浙江大学，中国）；Key Laboratory of Knowledge Engineering with Big Data (HFUT), Ministry of Education（大数据知识工程重点实验室（HFUT），教育部）；Institute of Artificial Intelligence, Hefei Comprehensive National Science Center, China（人工智能研究所，合肥综合性国家科学中心，中国）；Xinsight Lab, Research Institute, Hefei Zhongjuyuan Intelligent Technology Co., Ltd., China（Xinsight实验室，研究院，合肥中睿智能科技有限公司，中国）
Prototypical Calibrating Ambiguous Samples for Micro-Action Recognition
School of Computer Science and Information Engineering, Hefei University of Technology（合肥工业大学计算机与信息工程学院）；Institute of Artificial Intelligence, Hefei Comprehensive National Science Center（合肥综合性国家科学中心人工智能研究院）；ReLER Lab, CCAI, Zhejiang University（浙江大学CCAI ReLER实验室）
Hybrid-supervised Hypergraph-enhanced Transformer for Micro-gesture Based Emotion Recognition
School of Electronics and Information at Northwestern Polytechnical University（西北工业大学电子与信息学院）；Innovation Center NPU Chongqing（西北工业大学重庆创新中心）；University of Oulu（奥卢大学）
Spatial-Temporal Decoupled Adapter for Micro-gesture Online Recognition
Hefei University of Technology（合肥工业大学）；United Arab Emirates University（阿联酋大学）；Institute of Artificial Intelligence, Hefei Comprehensive National Science Center（人工智能研究院，合肥国家综合科学中心）；Anhui Evolution Technology Co., Ltd.（安徽进化科技有限公司）
Multi-Track Multimodal Learning on iMiGUE: Micro-Gesture and Emotion Recognition
Russian - Armenian University（俄罗斯-亚美尼亚大学）；Sber AI Lab（Sber AI实验室）；HSE University（俄罗斯高等经济大学）；Innopolis University（Innopolis大学）；ISP RAS Research Center for Trusted Artificial Intelligence（俄罗斯科学院可信人工智能研究中心）
University of Electronic Science and Technology of China（电子科技大学）；University of Cincinnati（辛辛那提大学）
Micro-drones can be integrated into various industrial applications but are constrained by their computing power and expert pilots, a secondary challenge. This study presents a computationally-efficient deep convolutional neural network that utilizes Gabor filters and spatial separable convolutions with low computational complexities. An attention module is integrated with the model to complement the performance. Further, perception-based action space and trajectory generators are integrated with the model's predictions for intuitive navigation. The computationally-efficient model aids a human…
Self-supervised Learning Matters: A Simple Ensemble Solution for Micro-Gesture Recognition
Hefei University of Technology（合肥工业大学）；United Arab Emirates University（阿拉伯联合酋长国大学）；Institute of Artificial Intelligence, Hefei Comprehensive National Science Center（合肥综合性国家科学中心人工智能研究院）；Anhui Evolution Technology Co., Ltd.（安徽进化科技有限公司）；Nanyang Technological University（南洋理工大学）；University College London（伦敦大学学院）；The University of Sydney（悉尼大学）；Beijing QBoson Quantum Technology Co., Ltd.（北京量子芯科技有限公司）
Active Inference for Micro-Gesture Recognition: EFE-Guided Temporal Sampling and Adaptive Learning
Tianjin Normal University（天津师范大学）；Shenzhen University（深圳大学）；The State Key Laboratory of Blockchain and Data Security, Zhejiang University（浙江大学区块链与数据安全国家重点实验室）；Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security（杭州高新技术区（滨江）区块链与数据安全研究院）；School of New Media and Communications, Tianjin University（天津大学新媒体与传播学院）
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Institute of Psychology, Chinese Academy of Sciences（中国科学院心理研究所）
Although automatic emotion recognition from facial expressions and speech has made remarkable progress, emotion recognition from body gestures has not been thoroughly explored. People often use a variety of body language to express emotions, and it is difficult to enumerate all emotional body gestures and collect enough samples for each category. Therefore, recognizing new emotional body gestures is critical for better understanding human emotions. However, the existing methods fail to accurately determine which emotional state a new body gesture belongs to. In order to solve this problem, we …
OMG-Bench: A New Challenging Benchmark for Skeleton-based Online Micro Hand Gesture Recognition
School of Systems Science and Engineering, Sun Yat-sen University（中山大学系统科学与工程学院）；Beijing University of Posts and Telecommunications（北京邮电大学）；Shanghai Jiao Tong University（上海交通大学）；Nankai University（南开大学）；Defense Innovation Institute, Academy of Military Sciences（军事科学院国防创新研究院）；Tianjin Artificial Intelligence Innovation Center（天津人工智能创新中心）
A Multi-Modal Framework with Cross-Subject Pseudo-Labeling and Semantic Alignment for Micro-Gesture Recognition
School of Computer Science and Information Engineering, Hefei University of Technology (HFUT)（合肥工业大学计算机科学与信息工程学院）；School of Computer Science, University of Auckland (UOA)（奥克兰大学计算机科学学院）
The emergence of depth imaging technologies like the Microsoft Kinect has renewed interest in computational methods for gesture classification based on videos. For several years now, researchers have used the Bag-of-Features (BoF) as a primary method for generation of feature vectors from video data for recognition of gestures. However, the BoF method is a coarse representation of the information in a video, which often leads to poor similarity measures between videos. Besides, when features extracted from different spatio-temporal locations in the video are pooled to create histogram vectors …
MSF-Mamba: Motion-aware State Fusion Mamba for Efficient Micro-Gesture Recognition
Computer Vision and Pattern Recognition Laboratory, Department of Computational Engineering, Lappeenranta-Lahti University of Technology LUT（计算机视觉与模式识别实验室，计算工程系，拉佩兰塔-拉赫蒂理工大学）；School of Electrical and Information Engineering, Tianjin University（电气与信息工程学院，天津大学）；School of Electrical and Electronic Engineering, Nanyang Technological University（电子与电气工程学院，南洋理工大学）；Faculty of Information Technology, Brno University of Technology（信息技术学院，布拉格技术大学）
K. J. Somaiya College of Engineering（K. J. 索马亚工程学院）；University of Mumbai（孟买大学）
This paper presents a system which can recognise hand poses & gestures from the Indian Sign Language (ISL) in real-time using grid-based features. This system attempts to bridge the communication gap between the hearing and speech impaired and the rest of the society. The existing solutions either provide relatively low accuracy or do not work in real-time. This system provides good results on both the parameters. It can identify 33 hand poses and some gestures from the ISL. Sign Language is captured from a smartphone camera and its frames are transmitted to a remote server for processing. The…
