Shanqing Cai, Subhashini Venugopalan, Katie Seaver, Xiang Xiao, Katrin Tomanek, Sri Jalasutram, Meredith Ringel Morris, Shaun Kane, Ajit Narayanan, Robert L. Ma… 展开作者
Shanqing Cai, Subhashini Venugopalan, Katie Seaver, Xiang Xiao, Katrin Tomanek, Sri Jalasutram, Meredith Ringel Morris, Shaun Kane, Ajit Narayanan, Robert L. MacDonald, Emily Kornman, Daniel Vance, Blair Casey, Steve M. Gleason, Philip Q. Nelson, Michael P. Brenner
Finding ways to accelerate text input for individuals with profound motor impairments has been a long-standing area of research. Closing the speed gap for augmentative and alternative communication (AAC) devices such as eye-tracking keyboards is important for improving the quality of life for such individuals. Recent advances in neural networks of natural language pose new opportunities for re-thinking strategies and user interfaces for enhanced text-entry for AAC users. In this paper, we present SpeakFaster, consisting of large language models (LLMs) and a co-designed user interface for text entry in a highly-abbreviated form, allowing saving 57% more motor actions than traditional predictive keyboards in offline simulation. A pilot study with 19 non-AAC participants typing on a mobile device by hand demonstrated gains in motor savings in line with the offline simulation, while introducing relatively small effects on overall typing speed. Lab and field testing on two eye-gaze typing users with amyotrophic lateral sclerosis (ALS) demonstrated text-entry rates 29-60% faster than traditional baselines, due to significant saving of expensive keystrokes achieved through phrase and word predictions from context-aware LLMs. These findings provide a strong foundation for further exploration of substantially-accelerated text communication for motor-impaired users and demonstrate a direction for applying LLMs to text-based user interfaces.
Designing Mobile Interaction Guidelines to Account for Situationally Induced Impairments and Disabilities (SIID) and Severely Constraining Situational Impairments (SCSI)
An AI-Driven Multimodal Smart Home Platform for Continuous Monitoring and Assistance in Post-Stroke Motor Impairment
University of Cambridge(剑桥大学);Beihang University(北京航空航天大学);KAUST(阿卜杜拉国王科技大学);Beijing New Guoxin Software Evaluation Technology Co ltd(北京新国信软件评测技术有限公司);Beijing Tsinghua Changgung Hospital, Tsinghua University(清华大学北京清华长庚医院);University College London(伦敦大学学院);Beijing Tongren Hospital, Capital Medical University(首都医科大学附属北京同仁医院)
A Sonomyography-based Muscle Computer Interface for Individuals with Spinal Cord Injury
Indian Institute of Technology Delhi(印度理工学院德里分校);George Mason University(乔治梅森大学);Indian Spinal Injuries Centre(印度脊髓损伤中心);All India Institute of Medical Sciences(全印医学科学学院)
Improved Dysarthric Speech to Text Conversion via TTS Personalization
Department of Telecommunications and Artificial Intelligence, Budapest University of Technology, Hungary(电信与人工智能系,布达佩斯技术大学);Division of Speech, Music and Hearing, KTH Royal Institute of Technology, Sweden(语音、音乐与听觉 division,皇家理工学院);Hungarian Research Centre for Linguistics, HUN-REN, Hungary(匈牙利语言研究与应用中心);SpeechTex Ltd., Hungary(SpeechTex有限公司);Institute of Informatics, University of Szeged, Hungary(信息学院,塞格德大学)