Evaluation of Pose Estimation Systems for Sign Language Translation
Catherine O'Brien, Gerard Sant, Mathias Müller, Sarah Ebling
备注:
Accepted at LREC 2026 Workshop on the Representation and Processing of Sign Languages. O'Brien and Sant contributed equally to this paper. 16 pages, 6 figures
Many sign language translation (SLT) systems operate on pose sequences instead of raw video to reduce input dimensionality, improve portability, and partially anonymize signers. The choice of pose estimator is often treated as an implementation detail, with systems defaulting to widely available tools such as MediaPipe Holistic or OpenPose. We present a systematic comparison of pose estimators for pose-based SLT, covering widely used baselines (MediaPipe Holistic, OpenPose) and newer whole-body/high-capacity models (MMPose WholeBody, OpenPifPaf, AlphaPose, SDPose, Sapiens, SMPLest-X). We quantify downstream impact by training a controlled SLT pipeline on RWTH-PHOENIX-Weather 2014 where only the pose representation varies, evaluating with BLEU and BLEURT.
To contextualize translation outcomes, we analyze temporal stability, missing hand keypoints, and robustness to occlusion using higher-resolution videos from the Signsuisse dataset. SDPose and Sapiens achieve the best translation performance (BLEU ~11.5), outperforming the common MediaPipe baseline (BLEU ~10). In occlusion cases, Sapiens is correct in all tested instances (15/15), while OpenPifPaf fails in nearly all (1/15) and also yields the weakest translation scores. Estimators that frequently leave out hand keypoints are associated with lower BLEU/BLEURT. We release code that can be used not only to reproduce our experiments, but also considerably lowers the barrier for other researchers to use alternative pose estimators.
University of Zurich(苏黎世大学);University of Dayton(Dayton 大学);University of Surrey(萨里大学);Idiap Research Institute(Idiap 研究院);EPFL(苏黎世联邦理工学院);Google DeepMind(谷歌DeepMind)
Enhanced Hallucination Detection in Neural Machine Translation through Simple Detector Aggregation
CentraleSupélec(中央高等电力学院);Universite Paris-Saclay(巴黎萨克雷大学);Inria(法国国家信息与自动化研究所);CEA(法国原子能和替代能源委员会);digeiz;Equall;Instituto de Telecomunicações(电信研究所);University of Lisbon(里斯本大学);Unbabel
Neural Machine Translation for Extremely Low-Resource African Languages: A Case Study on Bambara
Rochester Institute of Technology(罗切斯特理工学院);Centre National Collaboratif de l’Education en Robotique et en Intelligence Artificielle (RobotsMali)(国家机器人与人工智能教育合作中心(马里机器人));Google Research(谷歌研究院);Orange Silicon Valley(奥兰治硅谷)
Estonian WinoGrande Dataset: Comparative Analysis of LLM Performance on Human and Machine Translation
TartuNLP, Institute of Computer Science(塔尔图大学计算机科学研究所TartuNLP实验室);Department of Translation Studies, Institute of Foreign Language and Cultures(塔尔图大学外语与文化研究所翻译研究系);Institute of Genomics(塔尔图大学基因组学研究所);University of Tartu(塔尔图大学)
SignDATA: Data Pipeline for Sign Language Translation
Computer Science and Information Engineering National Central University(计算机科学与信息工程国家级中央大学);Electrical Engineering National Changhua University Of Education(电气工程国家彰化教育大学)