Over 5% of the world's population (466 million people) has disabling hearing loss. 4 million are children. They can be hard of hearing or deaf. Deaf people mostly have profound hearing loss. Which implies very little or no hearing. Over the world, deaf people often communicate using a sign language with gestures of both hands and facial expressions. The sign language is a full-fledged natural language with its own grammar and lexicon. Therefore, there is a need for translation models from and to sign languages. In this work, we are interested in the translation of Modern Standard Arabic(MSAr) into sign language. We generated a gloss representation from MSAr that extracts the features mandatory for the generation of animation signs. Our approach locates the most pertinent features that maintain the meaning of the input Arabic sentence.
Data Augmentation for Sign Language Gloss Translation
Bar-Ilan University(巴伊兰大学);Carnegie Mellon University(卡内基梅隆大学);Allen Institute for AI(艾伦人工智能研究所)
手语翻译(英语:Sign language translation,缩写为手语翻译)通常被分解为视频到注释识别和注释到文本翻译,其中注释是按照它们被签名的顺序转录的口语单词的序列。我们在这里专注于注释到文本的翻译,我们将其视为低资源神经机器翻译(NMT)问题。然而,与传统的低资源NMT不同,注释到文本的翻译不同,因为注释-文本对通常具有比口语对更高的词汇重叠和更低的句法重叠。我们利用这种词汇重叠和处理句法分歧,提出了两个基于规则的语法生成伪平行的单语口语文本的注释文本对。通过对由此获得的合成数据进行预训练,我们将美国手语(ASL)到英语和德国手语(DGS)到德语的翻译分别提高了3.14和2.20 BLEU。
Factorized Learning Assisted with Large Language Model for Gloss-free Sign Language Translation
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Macau University of Science and Technology(澳门科技大学);CAIR, HKISI, Chinese Academy of Sciences(中国科学院香港创新研究院);Mashang Consumer Finance(马上消费金融)
Cross-modality Data Augmentation for End-to-End Sign Language Translation
Thrust of Artificial Intelligence, HKUST (Guangzhou)(香港科技大学(广州)人工智能学科);Tencent AI Lab(腾讯人工智能实验室);Department of Computer Science and Engineering, HKUST(香港科技大学计算机科学与工程系);Guangzhou HKUST Fok Ying Tung Research Institute(广州香港科技大学霍英东研究院)
Introducing A Bangla Sentence - Gloss Pair Dataset for Bangla Sign Language Translation and Research
Department of Computer Science and Engineering, Bangladesh University of Engineering and Technology(Bangladesh University of Engineering and Technology计算机科学与工程系)
Selective Contrastive Learning For Gloss Free Sign Language Translation
School of Informatics, Xiamen University, China(厦门大学信息学院);Key Lab of Digital Protection and Intelligent Processing of Intangible Cultural Heritage of Fujian-Taiwan (XMU), Ministry of Culture and Tourism, China(福建省 Fujian-Taiwan 非物质文化遗产数字保护与智能处理重点实验室,文化部,中国);National Language Resources Monitoring and Research Center for Education and Teaching Media, Xiamen University, China(教育与教学媒体语言资源监测与研究中心,厦门大学,中国)
Enhanced Arabic Text Retrieval with Attentive Relevance Scoring
University of the Basque Country UPV/EHU(巴斯克国家大学UPV/EHU);Lab. de Genie Electrique (LAGE), University of Ouargla(电气工程实验室(LAGE),奥尔加拉大学);Faculty of Pharmacy, Helsinki University(药学院,赫尔辛基大学);IKERBASQUE, Basque Foundation for Science(ikerbasque,巴斯克科学基金会);Sorbonne University Abu Dhabi(索邦大学阿布扎比)
Grounding or Guessing? Visual Signals for Detecting Hallucinations in Sign Language Translation
German Research Center for Artificial Intelligence (DFKI GmbH)(德国人工智能研究中心(DFKI GmbH));Saarland Informatics Campus(萨尔兰州信息学校园);Barcelona Supercomputing Center (BSC-CNS)(巴塞罗那超级计算中心(BSC-CNS))
Breaking the Silence: A Dataset and Benchmark for Bangla Text-to-Gloss Translation
IIT, University of Dhaka, Bangladesh(达卡大学理工学院,孟加拉国);University of Maryland, Baltimore County, USA(马里兰大学巴尔的摩县分校,美国);SUNY, Old Westbury, USA(SUNY,美国奥尔德韦斯特伯里)
Improving Gloss-free Sign Language Translation by Reducing Representation Density
Artificial Intelligence Thrust, HKUST (Guangzhou)(香港科技大学(广州)人工智能方向);Tencent AI Lab(腾讯人工智能实验室);Department of Computer Science and Engineering, HKUST(香港科技大学计算机科学与工程系);Guangzhou HKUST Fok Ying Tung Research Institute(广州香港科技大学霍英东研究院)
Ara-HOPE: Human-Centric Post-Editing Evaluation for Dialectal Arabic to Modern Standard Arabic Translation
School of Informatics, University of Edinburgh(爱丁堡大学信息学院);Leids Universitair Medisch Centrum & LIACS, Universiteit Leiden(莱顿大学莱顿医学校区及LIACS);School of Computer Science, University of Manchester(曼彻斯特大学计算机科学学院)
C${^2}$RL: Content and Context Representation Learning for Gloss-free Sign Language Translation and Retrieval
State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS), Institute of Automation, Chinese Academy of Sciences (CASIA)(中国科学院自动化研究所多模态人工智能系统国家重点实验室);School of Artificial Intelligence, University of Chinese Academy of Sciences (UCAS)(中国科学院大学人工智能学院);Macau University of Science and Technology(澳门科技大学);NIO(蔚来汽车)
The Landscape of Arabic Large Language Models (ALLMs): A New Era for Arabic Language Technology
is a Researcher at iWAN Research Group, King Saud University(iWAN研究组,国王沙特大学);is a Senior Scientist at Qatar Computing Research Institute(卡塔尔计算研究所);is a Professor at King Saud University(国王沙特大学);Head of iWAN Research Group(iWAN研究组)