Transformer 时间头和池化后运动门控对基于 CorrNet 的连续手语识别有帮助吗？实证研究
Do Transformer Temporal Heads and Post-Pooling Motion Gates Help CorrNet-based CSLR? An Empirical Study
浏览论文内容
中文总结 AI 辅助
研究 CorrNet 系统的两种扩展，即换用 Transformer 头和增加 MotionGate 模块用于连续手语识别。发现 Transformer 头未超 BiLSTM 基线，MotionGate 效果不佳，表明相关架构扩展应谨慎测试，不能想当然认为有帮助。
中文摘要 AI 辅助
CorrNet 是连续手语识别（CSLR）的强大基线，因其能在视觉编码阶段对帧间相关性建模。本文研究了重现的 CorrNet 系统的两种自然扩展：用 Transformer 编码器替换 BiLSTM 时间头，以及在时间池化后注入运动线索。发现即便调整训练策略，Transformer 头也未超越 BiLSTM 基线，且二者计算和运行成本相近。对于第二种扩展，设计了名为 MotionGate 的轻量级模块。实验中，MotionGate 常退化为恒等映射，门控失去运动选择性，注入的残差成为池化特征的微弱、非选择性扰动。这些结果表明 CorrNet 基于相关性编码后的显式运动注入大多冗余，CSLR 中看似自然的架构扩展应谨慎测试而非假定有帮助。
英文摘要
CorrNet is a strong baseline for continuous sign language recognition (CSLR) because it models inter-frame correlations inside the visual encoding stage. In this paper, we study two natural extensions of a reproduced CorrNet system: replacing the BiLSTM temporal head with a Transformer encoder, and injecting motion cues after temporal pooling. We find that the Transformer head does not outperform the BiLSTM baseline, even with a training strategy adjusted for the Transformer, and the two heads have almost the same computational and runtime cost. For the second extension, we design a lightweight module called MotionGate. In our experiments, MotionGate consistently collapses to an identity-like mapping: the gate loses motion selectivity, and the injected residual becomes a weak, non-selective perturbation of the pooled features. These results suggest that explicit motion injection after CorrNet's correlation-based encoding is largely redundant, and that natural-looking architectural extensions in CSLR should be tested carefully instead of being assumed to help.
发表机构
- University of New South Wales（新南威尔士大学）
- National Centre for Computer Animation, Faculty of Media, Science and Technology, Bournemouth University（伯恩茅斯大学媒体、科学与技术学院国家计算机动画中心）
- School of Information Science and Engineering, Dalian Polytechnic University（大连工业大学信息科学与工程学院）
机构由 AI 辅助整理，请以论文原文为准。
