Skeleton-based temporal action segmentation is a fundamental yet challenging task, playing a crucial role in enabling intelligent systems to perceive and respond to human activities. While fully-supervised methods achieve satisfactory performance, they require costly frame-level annotations and are sensitive to ambiguous action boundaries. To address these issues, we introduce a point-supervised framework for skeleton-based action segmentation, where only a single frame per action segment is labeled. We leverage multimodal skeleton data, including joint, bone, and motion information, encoded via a pretrained unified model to extract rich feature representations. To generate reliable pseudo-labels, we propose a novel prototype similarity method and integrate it with two existing methods: energy function and constrained K-Medoids clustering. Multimodal pseudo-label integration is proposed to enhance the reliability of the pseudo-label and guide the model training. We establish new benchmarks on PKU-MMD (X-Sub and X-View), MCFS-22, and MCFS-130, and implement baselines for point-supervised skeleton-based human action segmentation. Extensive experiments show that our method achieves competitive performance, even surpassing some fully-supervised methods while significantly reducing annotation effort.
Skeleton Motion Words for Unsupervised Skeleton-Based Temporal Action Segmentation
University of Bonn, Germany(波恩大学);University College London, UK(伦敦大学学院);Lamarr Institute for Machine Learning and Artificial Intelligence, Germany(拉马尔人工智能与机器学习研究所)
Slender Object Scene Segmentation in Remote Sensing Image Based on Learnable Morphological Skeleton with Segment Anything Model
School of Mathematical Sciences, Beijing Normal University(北京师范大学数学科学学院);State Key Laboratory of Remote Sensing Science, Faculty of Geographical Science, Beijing Normal University(北京师范大学地理科学学部遥感科学国家重点实验室)
SkelGen4D: Weakly-Supervised Skeleton-Based 4D Generation for Text-Driven Mesh Animation
School of Data Science, Lingnan University(岭南大学数据科学学院);Department of Computer Science and Engineering, Chinese University of Hong Kong(香港中文大学计算机科学与工程系);Autodesk Research(Autodesk研究院);Institute of Computer Applications, Shanghai Jiao Tong University(上海交通大学计算机应用研究所)
University of Science and Technology of China(中国科学技术大学);School of Biomedical Engineering(生物医学工程学院);Naval Medical University(海军军医大学);Second Affiliated Hospital(第二附属医院);Suzhou Aerospace Information Research Institute(苏州航天信息研究院)
Bootstrapped Representation Learning for Skeleton-Based Action Recognition
Lund University(隆德大学);Sony Europe B.V.(索尼欧洲有限公司)
在这项工作中,我们研究了自监督表示学习的三维基于动作识别。我们扩展了Bootstrap Your Own Latent(BYOL)算法用于骨架序列数据的表示学习,并提出了一种新的数据增强策略,该策略包括两个非对称的转换管道。我们还介绍了一种多视点采样方法,利用不同的相机捕捉到的同一动作的多个视角。在半监督设置,我们表明,性能可以进一步提高知识蒸馏从更广泛的网络,再次利用未标记的样本。我们在NTU-60和NTU-120数据集上进行了大量的实验,以证明我们所提出的方法的性能。我们的方法在线性评估和半监督基准测试中始终优于当前最先进的方法。
Self-supervised Action Representation Learning from Partial Spatio-Temporal Skeleton Sequences
Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学高瓴人工智能学院);The Chinese University of Hong Kong(香港中文大学);Shanghai AI Laboratory(上海人工智能实验室)
Cascaded multitask U-Net using topological loss for vessel segmentation and centerline extraction
Université de Reims Champagne Ardenne(兰斯香槟阿登大学);Univ Lyon(里昂大学);INSA-Lyon(里昂国立应用科学学院);Université Claude Bernard Lyon 1(里昂第一大学);UJM-Saint Etienne(圣埃蒂安大学);CNRS(法国国家科学研究中心);Inserm(法国国家健康与医学研究院);CREATIS(CREATIS研究院)
Self-Supervised Gait Encoding with Locality-Aware Attention for Person Re-Identification
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院);South China University of Technology(华南理工大学);The Chinese University of Hong Kong(香港中文大学);National University of Defense Technology(国防科技大学);Lanzhou University(兰州大学)
Spectral Scalpel: Amplifying Adjacent Action Discrepancy via Frequency-Selective Filtering for Skeleton-Based Action Segmentation
Harbin Institute of Technology, Shenzhen(哈尔滨工业大学(深圳));Shenzhen H&T Intelligent Control Co., Ltd.(深圳H&T智能控制有限公司);Southern University of Science and Technology(南方科技大学);Southeast University(东南大学)
Language-Assisted Human Part Motion Learning for Skeleton-Based Temporal Action Segmentation
State Key Laboratory of Robotics and Systems, Harbin Institute of Technology Shenzhen(哈尔滨工业大学深圳校区机器人学国家重点实验室);e-Media Research Lab, KU Leuven(鲁汶大学电子媒体研究实验室);Peng Cheng Laboratory(鹏城实验室)
Enhancing Boundary Segmentation for Topological Accuracy with Skeleton-based Methods
University of Science and Technology Beijing(北京科技大学);Beijing Advanced Innovation Center for Materials Genome Engineering(北京材料基因组创新中心);School of Intelligence Science and Technology(智能科学与技术学院);Shunde Innovation School(顺德创新学校);Institute for Advanced Materials and Technology(先进材料与技术研究院);Key Laboratory of Intelligent Bionic Unmanned Systems(智能仿生无人系统重点实验室);Institute of Materials Intelligent Technology(材料智能技术研究院);Liaoning Academy of Materials(辽宁省材料科学院);School of Materials Science and Technology(材料科学与技术学院)
SM-SGE: A Self-Supervised Multi-Scale Skeleton Graph Encoding Framework for Person Re-Identification
Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院);The Chinese University of Hong Kong(香港中文大学);Lanzhou University(兰州大学);Beijing Institute of Technology(北京理工大学)
Exploring Adaptive Masked Reconstruction for Self-Supervised Skeleton-Based Action Recognition
Hefei University of Technology(合肥工业大学);Jilin University(吉林大学);Zhejiang Gongshang University(浙江工商大学);University of Science and Technology of China(中国科学技术大学)
A Self-Supervised Gait Encoding Approach with Locality-Awareness for 3D Skeleton Based Person Re-Identification
Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院);National University of Defense Technology(国防科技大学);Lanzhou University(兰州大学);South China University of Technology(华南理工大学);Second Clinical Medical College, Jinan University(暨南大学第二临床医学院);Beijing Institute of Technology(北京理工大学)
Contrastive Learning from Extremely Augmented Skeleton Sequences for Self-supervised Action Recognition
Key Laboratory of Machine Perception, Shenzhen Graduate School, Peking University(北京大学深圳研究生院机器感知重点实验室);The School of Intelligent Systems Engineering, Sun Yat-sen University(中山大学智能系统工程学院)
近年来,随着对比学习方法的发展,用于基于手势的动作识别的自监督表示学习得到了发展。现有的对比学习方法使用正常的增强来构造相似的正样本,这限制了探索新的运动模式的能力。在本文中,更好地利用极端扩增引入的运动模式,一个对比学习框架,利用丰富的信息挖掘的自我监督的行动表示(Aimmatis)的建议。首先,提出了极端增强和基于能量的注意力引导下降模块(EADM)来获得不同的正样本,这带来了新的运动模式,以提高学习表示的通用性。其次,由于原始恒等式的剧烈变化,直接使用极端增广可能无法提高性能,因此提出了双分布发散最小化损失(Dual DistributionalDivergenceMinimizationLoss,$^3$ M Loss),以更温和的方式最小化分布发散。第三,提出了最近邻挖掘(NNM),进一步扩大正样本,使丰富的信息挖掘过程更加合理。在NTU RGB+D 60,PKU-MMD,NTU RGB+D 120数据集上的详尽实验已经验证了我们的Aimestant可以在各种评估协议下显著优于最先进的方法,并观察到更高质量的动作表示。我们的代码可在https://github.com/Levigty/AimCLR上获得。