This review summarises the status of silent speech interface (SSI) research. SSIs rely on non-acoustic biosignals generated by the human body during speech production to enable communication whenever normal verbal communication is not possible or not desirable. In this review, we focus on the first case and present latest SSI research aimed at providing new alternative and augmentative communication methods for persons with severe speech disorders. SSIs can employ a variety of biosignals to enable silent communication, such as electrophysiological recordings of neural activity, electromyographic (EMG) recordings of vocal tract movements or the direct tracking of articulator movements using imaging techniques. Depending on the disorder, some sensing techniques may be better suited than others to capture speech-related information. For instance, EMG and imaging techniques are well suited for laryngectomised patients, whose vocal tract remains almost intact but are unable to speak after the removal of the vocal folds, but fail for severely paralysed individuals. From the biosignals, SSIs decode the intended message, using automatic speech recognition or speech synthesis algorithms. Despite considerable advances in recent years, most present-day SSIs have only been validated in laboratory settings for healthy users. Thus, as discussed in this paper, a number of challenges remain to be addressed in future research before SSIs can be promoted to real-world applications. If these issues can be addressed successfully, future SSIs will improve the lives of persons with severe speech impairments by restoring their communication capabilities.
Poster: Recognizing Hidden-in-the-Ear Private Key for Reliable Silent Speech Interface Using Multi-Task Learning
The University of Tokyo（东京大学）；The Chinese University of Hong Kong（香港中文大学）；Dalian Maritime University（大连海事大学）；University of Toronto（多伦多大学）；Tsinghua University（清华大学）
Harnessing AI for Speech Reconstruction using Multi-view Silent Video Feed
Adobe Systems（奥多比系统公司）；Netaji Subhas Institute of Technology（内塔吉·苏巴斯理工学院）；National Institute of Informatics（国立信息学研究所）；National University of Singapore（新加坡国立大学）
Adaptation of Tongue Ultrasound-Based Silent Speech Interfaces Using Spatial Transformer Networks
University of Szeged（塞格德大学）；ELRN-SZTE Research Group on Artificial Intelligence（ELRN-塞格德大学人工智能研究组）；Budapest University of Technology and Economics（布达佩斯技术与经济大学）
Ultrasound-based Silent Speech Interface Built on a Continuous Vocoder
Budapest University of Technology and Economics（布达佩斯技术与经济大学）；MTA-SZTE Research Group on Artificial Intelligence（匈牙利科学院-塞格德大学人工智能研究组）；University of Szeged（塞格德大学）
Affect Decoding in Phonated and Silent Speech Production from Surface EMG
1 CHI -- Chair of Health Informatics, TUM University Hospital, Munich, Germany 2 SAIL -- Signal Analysis；Interpretation Lab, University of Southern California, USA 3 CSL -- Cognitive Systems Lab, University of Bremen, Germany 4 GLAM -- Group on Language, Audio, \& Music, Imperial College London, UK 5 MCML -- Munich Center for Machine Learning, Germany
Cross-Modal Masking for Robust Silent Speech Synthesis Using sEMG and Lipreading
Aholab research group within the HiTZ Center at University of the Basque Country (UPV/EHU)（巴斯克大学HiTZ中心内Aholab研究组）；PRHLT research center, Universitat Politècnica de València (UPV)（瓦伦西亚理工大学PRHLT研究中心）
Autoencoder-Based Articulatory-to-Acoustic Mapping for Ultrasound Silent Speech Interfaces
MTA-SZTE Research Group on Artificial Intelligence（MTA-SZTE人工智能研究组）；University of Szeged（塞格德大学）；Eötvös Loránd University（厄特沃什·罗兰大学）；Budapest University of Technology and Economics（布达佩斯技术与经济大学）
IR-UWB Radar-Based Contactless Silent Speech Recognition of Vowels, Consonants, Words, and Phrases
School of Integrated Technology, Yonsei University（延世大学综合技术学院）；NVIDIA Corporation（英伟达公司）；Department of Convergence IT Engineering, Pohang University of Science and Technology（浦项科技大学融合IT工程系）
Improving Neural Silent Speech Interface Models by Adversarial Training
University of Szeged（塞格德大学）；MTA-SZTE Research Group on Artificial Intelligence（匈牙利科学院-塞格德大学人工智能研究组）；Budapest University of Technology and Economics（布达佩斯技术与经济大学）；Eötvös Loránd University（罗兰大学）
Silent Speech Interfaces in the Era of Large Language Models: A Comprehensive Taxonomy and Systematic Review
National University of Defense Technology（国防科技大学）；College of Computer Science and Technology, National University of Defense Technology（国防科技大学计算机科学与技术学院）；National Key Laboratory of Parallel and Distributed Computing, College of Computer Science and Technology, National University of Defense Technology（国防科技大学计算机科学与技术学院并行与分布式计算国家重点实验室）；State Key Laboratory of Complex & Critical Software Environment, National University of Defense Technology（国防科技大学复杂与关键软件环境国家重点实验室）；Hunan Normal University（湖南师范大学）；College of Computer Science, Hunan Normal University（湖南师范大学计算机学院）；Hunan University（湖南大学）；College of Computer Science, Hunan University（湖南大学计算机学院）
