基于运动注意力机制与帧级自蒸馏的连续手语识别
Continuous Sign Language Recognition Based on Motor attention mechanism and frame-level Self-distillation
AI总结：
针对连续手语识别忽视动态变化的问题，提出结合运动注意力与帧级自蒸馏的 MAM-FSD 模型，在三个公开数据集上提升准确率并达到最优水平。
摘要：
面部表情、头部运动、身体运动和手势运动的变化是手语识别中的显著线索，而当前大多数连续手语识别（CSLR）研究方法在帧级特征提取阶段主要关注视频序列中的静态图像，忽视了图像中的动态变化。本文提出一种新颖的运动注意力机制，用于捕捉手语表达过程中局部运动区域的形变，并获得图像变化的动态表示。同时，我们首次将自蒸馏方法应用于连续手语的帧级特征提取：通过对相邻阶段的特征进行自蒸馏，并利用高阶特征作为教师来指导低阶特征，在不增加计算资源的情况下提升特征表达能力。二者结合构成了我们提出的基于运动注意力机制和帧级自蒸馏的 CSLR 整体模型 MAM-FSD，提升了模型的推理能力与鲁棒性。我们在三个公开数据集上进行实验，实验结果表明，所提方法能够有效提取视频中的手语运动信息，提高 CSLR 的准确率，并达到当前最优水平。
英文摘要：
Changes in facial expression, head movement, body movement and gesture movement are remarkable cues in sign language recognition, and most of the current continuous sign language recognition(CSLR) research methods mainly focus on static images in video sequences at the frame-level feature extraction stage, while ignoring the dynamic changes in the images. In this paper, we propose a novel motor attention mechanism to capture the distorted changes in local motion regions during sign language expression, and obtain a dynamic representation of image changes. And for the first time, we apply the self-distillation method to frame-level feature extraction for continuous sign language, which improves the feature expression without increasing the computational resources by self-distilling the features of adjacent stages and using the higher-order features as teachers to guide the lower-order features. The combination of the two constitutes our proposed holistic model of CSLR Based on motor attention mechanism and frame-level Self-Distillation (MAM-FSD), which improves the inference ability and robustness of the model. We conduct experiments on three publicly available datasets, and the experimental results show that our proposed method can effectively extract the sign language motion information in videos, improve the accuracy of CSLR and reach the state-of-the-art level.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
