Skeleton-based action recognition, which classifies human actions based on the coordinates of joints and their connectivity within skeleton data, is widely utilized in various scenarios. While Graph Convolutional Networks (GCNs) have been proposed for skeleton data represented as graphs, they suffer from limited receptive fields constrained by joint connectivity. To address this limitation, recent advancements have introduced transformer-based methods. However, capturing correlations between all joints in all frames requires substantial memory resources. To alleviate this, we propose a novel approach called Skeletal-Temporal Transformer (SkateFormer) that partitions joints and frames based on different types of skeletal-temporal relation (Skate-Type) and performs skeletal-temporal self-attention (Skate-MSA) within each partition. We categorize the key skeletal-temporal relations for action recognition into a total of four distinct types. These types combine (i) two skeletal relation types based on physically neighboring and distant joints, and (ii) two temporal relation types based on neighboring and distant frames. Through this partition-specific attention strategy, our SkateFormer can selectively focus on key joints and frames crucial for action recognition in an action-adaptive manner with efficient computation. Extensive experiments on various benchmark datasets validate that our SkateFormer outperforms recent state-of-the-art methods.
Leveraging Foundation Model Automatic Data Augmentation Strategies and Skeletal Points for Hands Action Recognition in Industrial Assembly Lines
Faculty of Robot Science and Engineering, Northeastern University（东北大学机器人科学与工程学院）；Foshan Graduate School, Northeastern University（东北大学佛山研究生院）；College of Information Science and Engineering, Northeastern University（东北大学信息科学与工程学院）；The State Key Laboratory of Synthetical Automation for Process Industries, Northeastern University（东北大学流程工业综合自动化国家重点实验室）
Frequency Guidance Matters: Skeletal Action Recognition by Frequency-Aware Mixed Transformer
University of North Carolina at Charlotte（北卡罗来纳大学夏洛特分校）；Carnegie Mellon University（卡内基梅隆大学）；Microsoft Corporation（微软公司）；University of Central Florida（中佛罗里达大学）
A Deep Learning Approach for Real-Time 3D Human Action Recognition from Skeletal Data
Cerema（法国路桥与道路研究中心）；Université Toulouse III - Paul Sabatier（图卢兹第三大学（保罗·萨巴蒂埃大学））；Aparnix；Cortexica Vision Systems Ltd.（Cortexica视觉系统有限公司）；Queen Mary University of London（伦敦玛丽女王大学）；University Carlos III of Madrid（马德里卡洛斯三世大学）
Representation-Centric Survey of Supervised Skeletal Action Recognition and the New Benchmark
School of Computing, Australian National University（澳大利亚国立大学计算学院）；University of Alabama at Birmingham（阿拉巴马大学伯明翰分校）；OPPO US Research Center（OPPO美国研究中心）；Carnegie Mellon University（卡内基梅隆大学）；University of Western Australia（西澳大利亚大学）；Curtin University（Curtin大学）；School of Engineering and Built Environment, Griffith University（格里菲斯大学工程与环境学院）；School of Medicine, Yale University（耶鲁大学医学院）
Skeletal Movement to Color Map: A Novel Representation for 3D Action Recognition with Inception Residual Networks
Carlos III University of Madrid（马德里卡洛斯三世大学）；Cerema Research Center（Cerema研究中心）；Institut de Recherche en Informatique de Toulouse（图卢兹计算机科学研究所）；Université de Toulouse（图卢兹大学）
Decoupled Spatial-Temporal Attention Network for Skeleton-Based Action Recognition
University of Chinese Academy of Sciences（中国科学院大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；CAS Center for Excellence in Brain Science and Intelligence Technology（中国科学院脑科学与智能技术卓越创新中心）
Temporal-Viewpoint Transportation Plan for Skeletal Few-shot Action Recognition
Australian National University（澳大利亚国立大学）；Data61/CSIRO（澳大利亚联邦科学与工业研究组织数据61研究所）
我们提出了一个少样本学习管道的联合tEmporal和cAmera的视点算法（JEANIE）的三维基于动作的动作识别。为了排除查询和支持序列之间的3D身体关节的不对齐，我们提出了一个先进的动态时间规整的变种，它联合建模查询和支持帧之间的每个平滑路径，同时实现最佳的对齐时间和模拟相机的视点空间有限的Few-Shot训练数据下的端到端学习。基于简单谱图卷积（一种轻量级线性图神经网络骨干）的时间块编码器对序列进行编码。我们还包括一个带有Transformer的设置。最后，我们提出了一个基于相似性的损失，鼓励对齐的序列相同的类，同时防止不相关的序列的对齐。我们展示了NTU-60，NTU-120，Kinetics骨架和UWA 3D Multiview Activity II的最新结果。
TASAR: Transfer-based Attack on Skeletal Action Recognition
Hefei University of Technology（合肥工业大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Beijing Academy of Artificial Intelligence（北京智源人工智能研究院）；Beihang University（北京航空航天大学）；University College London（伦敦大学学院）
Motion Matters: Motion-guided Modulation Network for Skeleton-based Micro-Action Recognition
University College London（伦敦大学学院）；ReLER, CCAI, Zhejiang University（ReLER、CCAI、浙江大学）；Hefei University of Technology（合肥工业大学）；Institute of Artificial Intelligence, Hefei Comprehensive National Science Center（人工智能研究院、合肥综合性国家科学中心）；Intelligent Interconnected Systems Laboratory of Anhui Province（安徽省智能互联系统实验室）
Skeleton Based Action Recognition using a Stacked Denoising Autoencoder with Constraints of Privileged Information
Hefei University（合肥学院）；School of Artificial Intelligence and Big Data, Hefei University（合肥学院人工智能与大数据学院）；Institute of Applied Optimization, School of Artificial Intelligence and Big Data, Hefei University（合肥学院人工智能与大数据学院应用优化研究所）；School of Computer Science and Technology, University of Science and Technology of China（中国科学技术大学计算机科学与技术学院）；University of Science and Technology of China（中国科学技术大学）
Informative Sample Selection Model for Skeleton-based Action Recognition with Limited Training Samples
State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University（信息工程测绘遥感国家重点实验室，武汉大学）；Information Systems Technology and Design Pillar, Singapore University of Technology and Design（信息系统技术与设计 pillar，新加坡科技设计大学）；Shanghai Academy of AI for Science, Shanghai, China（上海人工智能科学研究院，上海，中国）；Department of Computer Science and Engineering, University at Buffalo, State University of New York（计算机科学与工程系，布法罗大学，纽约州立大学）；School of Computer Science, Wuhan University（武汉大学计算机学院）
School of Computer Science and Engineering, Nanjing University of Science and Technology（南京理工大学计算机科学与工程学院）；Artificial Intelligence Industrial Technology Research Institute, Nanjing Institute of Technology（南京工程学院人工智能产业技术研究院）；School of Computer Science, Nanjing University（南京大学计算机科学学院）
针对无监督骨骼动作识别中随机增强难以产生直接语义扰动且会软化正负对的问题，提出 A²MC，利用攻击增强生成困难正样本并由 PNM 混合生成困难负样本，在三个公开数据集上达到与 SOTA 相当的性能。
HDBN: A Novel Hybrid Dual-branch Network for Robust Skeleton-based Action Recognition
School of Intelligent Systems Engineering, Sun Yat-sen University（中山大学智能工程学院）；National Key Laboratory of General Artificial Intelligence, Peking University（北京大学通用人工智能重点实验室）
