拼接、对比与分割：利用修剪后的骨架视频学习人体动作分割模型
Stitch Contrast and Segment_Learning a Human Action Segmentation Model Using Trimmed Skeleton Videos
AI总结：
本文提出一种基于修剪骨架视频训练、在未修剪视频上测试的人体动作分割框架，通过拼接、对比和分割三步法实现无需逐帧标注的端到端动作定位与分类。
摘要：
现有的基于骨架的人体动作分类模型在训练和测试时都依赖于经过良好修剪的特定动作骨架视频，这阻碍了它们在以展示连续动作的未修剪视频为主的真实世界应用中的可扩展性。为了克服这一限制，最近引入的骨架动作分割模型将未修剪的骨架视频纳入端到端训练中。该模型经过优化，能够对任意长度的测试视频提供逐帧预测，同时实现动作定位和分类。然而，实现这样的改进需要逐帧标注的骨架视频，这在实践中仍然非常耗时。本文提出了一种用于基于骨架动作分割的新颖框架，该框架在短修剪骨架视频上进行训练，但能够在更长的未修剪视频上运行。该方法分三个步骤实现：拼接、对比和分割。首先，拼接提出了一种时间骨架拼接方案，将修剪后的骨架视频视为构成语义空间的基本人体动作，并可以对其进行采样以生成多动作拼接序列。对比通过一种新颖的判别前置任务从拼接序列中学习对比表示，使骨架编码器能够学习有意义的动作时间上下文以改善动作分割。最后，分割通过学习分割层并处理特定的数据可用性，将所提出的方法与动作分割联系起来。实验在适应公式中涉及一个修剪的源数据集和一个未修剪的目标数据集，用于真实世界基于骨架的人体动作分割，以评估所提出方法的有效性。
英文摘要：
Existing skeleton-based human action classification models rely on well-trimmed action-specific skeleton videos for both training and testing, precluding their scalability to real-world applications where untrimmed videos exhibiting concatenated actions are predominant. To overcome this limitation, recently introduced skeleton action segmentation models involve un-trimmed skeleton videos into end-to-end training. The model is optimized to provide frame-wise predictions for any length of testing videos, simultaneously realizing action localization and classification. Yet, achieving such an improvement im-poses frame-wise annotated skeleton videos, which remains time-consuming in practice. This paper features a novel framework for skeleton-based action segmentation trained on short trimmed skeleton videos, but that can run on longer un-trimmed videos. The approach is implemented in three steps: Stitch, Contrast, and Segment. First, Stitch proposes a tem-poral skeleton stitching scheme that treats trimmed skeleton videos as elementary human motions that compose a semantic space and can be sampled to generate multi-action stitched se-quences. Contrast learns contrastive representations from stitched sequences with a novel discrimination pretext task that enables a skeleton encoder to learn meaningful action-temporal contexts to improve action segmentation. Finally, Segment relates the proposed method to action segmentation by learning a segmentation layer while handling particular da-ta availability. Experiments involve a trimmed source dataset and an untrimmed target dataset in an adaptation formulation for real-world skeleton-based human action segmentation to evaluate the effectiveness of the proposed method.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
