Fingerspelling PoseNet：基于姿态的 Transformer 模型增强手指拼写翻译
Fingerspelling PoseNet: Enhancing Fingerspelling Translation with Pose-Based Transformer Models
AI总结：
针对美国手语手指拼写翻译任务，提出基于手部姿态估计的 Transformer 编码器-解码器架构，结合预测单词长度的新损失项与两阶段推理重排序方法，在 ChicagoFSWild 和 ChicagoFSWild+ 上实现超 10% 的相对性能提升。
摘要：
我们处理了利用自然场景视频进行美国手语手指拼写翻译的任务。我们利用了更精确的手部姿态估计方面的进展，提出了一种新颖的架构，该架构利用基于 Transformer 的编码器-解码器模型，实现无缝的上下文单词翻译。该翻译模型通过一个新颖的损失项进行增强，该损失项能准确预测手指拼写单词的长度，有利于训练和推理。我们还提出了一种新颖的两阶段推理方法，利用解码器的语言模型能力对假设进行重新排序。通过大量实验，我们证明我们提出的方法在 ChicagoFSWild 和 ChicagoFSWild+ 上优于现有最先进的模型，实现了超过 10% 的相对性能提升。我们的发现突出了我们方法的有效性及其在推进手语翻译中手指拼写识别方面的潜力。代码可在 https://github.com/pooyafayyaz/Fingerspelling-PoseNet 获取。
英文摘要：
We address the task of American Sign Language fingerspelling translation using videos in the wild. We exploit advances in more accurate hand pose estimation and propose a novel architecture that leverages the transformer based encoder-decoder model enabling seamless contextual word translation. The translation model is augmented by a novel loss term that accurately predicts the length of the finger-spelled word, benefiting both training and inference. We also propose a novel two-stage inference approach that re-ranks the hypotheses using the language model capabilities of the decoder. Through extensive experiments, we demonstrate that our proposed method outperforms the state-of-the-art models on ChicagoFSWild and ChicagoFSWild+ achieving more than 10% relative improvement in performance. Our findings highlight the effectiveness of our approach and its potential to advance fingerspelling recognition in sign language translation. Code is also available at https://github.com/pooyafayyaz/Fingerspelling-PoseNet.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
