CLIP-SLA：用于连续手语识别的参数高效CLIP适应
CLIP-SLA: Parameter-Efficient CLIP Adaptation for Continuous Sign Language Recognition
AI总结：
提出CLIP-SLA框架，通过参数高效微调将CLIP视觉编码器适应于连续手语识别，在四个数据集上以更少参数超越SOTA。
摘要：
连续手语识别（CSLR）专注于解释和转录视频中的手语手势序列。在这项工作中，我们提出了CLIP手语适应（CLIP-SLA），这是一种新颖的CSLR框架，通过参数高效微调（PEFT）利用CLIP模型中强大的预训练视觉编码器来适应手语任务。我们引入了两种变体，SLA-Adapter和SLA-LoRA，它们将PEFT模块集成到CLIP视觉编码器中，实现了以极少的可训练参数进行微调。所提框架的有效性在四个数据集上得到了验证：Phoenix2014、Phoenix2014-T、CSL-Daily和Isharah-500，其中两种CLIP-SLA变体均以更少的可训练参数优于几种SOTA模型。广泛的消融研究强调了所提方法在不同视觉-语言模型下用于CSLR的有效性和灵活性。这些发现展示了适应大规模预训练模型以实现可扩展和高效CSLR的潜力，为手语理解的未来发展铺平了道路。
英文摘要：
Continuous sign language recognition (CSLR) focuses on interpreting and transcribing sequences of sign language gestures in videos. In this work, we propose CLIP sign language adaptation (CLIP-SLA), a novel CSLR framework that leverages the powerful pre-trained visual encoder from the CLIP model to sign language tasks through parameter-efficient fine-tuning (PEFT). We introduce two variants, SLA-Adapter and SLA-LoRA, which integrate PEFT modules into the CLIP visual encoder, enabling fine-tuning with minimal trainable parameters. The effectiveness of the proposed frameworks is validated on four datasets: Phoenix2014, Phoenix2014-T, CSL-Daily, and Isharah-500, where both CLIP-SLA variants outperformed several SOTA models with fewer trainable parameters. Extensive ablation studies emphasize the effectiveness and flexibility of the proposed methods with different vision-language models for CSLR. These findings showcase the potential of adapting large-scale pre-trained models for scalable and efficient CSLR, which pave the way for future advancements in sign language understanding.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
