SpeechLLMs are increasingly deployed in professional settings where domain customisation is standard practice: users supply context in prompts with sensitive information, fine-tune on proprietary recordings, or both. We identify and systematically investigate an overlooked privacy risk of such customisation: a model adapted to recognise domain-specific terminology can be nudged into transcribing a phonetically similar word from its context or training data, even when a different word is spoken, thereby leaking private information. To evaluate this risk, we propose a technique to automatically construct benchmarks of such attacks and apply it to measure leakage rates across two customisation mechanisms, prompting and fine-tuning. Both mechanisms cause measurable leakage, compounding when combined. We evaluate a prompt-level mitigation strategy and analyse the accuracy-leakage trade-off across customisation approaches, finding that fine-tuning without context prompts offers the best balance.
Personal Attribute Leakage in Federated Speech Models
Mohamed bin Zayed University of Artificial Intelligence(Mohamed bin Zayed人工智能大学);Maastricht University(马斯特里赫特大学);University of Groningen(格罗宁根大学);University of Edinburgh(爱丁堡大学)
Out of the Box, into the Clinic? Evaluating State-of-the-Art ASR for Clinical Applications for Older Adults
Department of Public Health and Primary Care, Leiden University Medical Center(莱顿大学医学中心公共卫生与初级保健系);Leiden Institute of Advanced Computer Science, Leiden University(莱顿大学高级计算机科学研究所);Center for Medicine for Older People, Leiden University Medical Center(莱顿大学医学中心老年医学中心);Department of Internal Medicine, Leiden University Medical Center(莱顿大学医学中心内科系)
EIA: Environmental Injection Attack on Generalist Web Agents for Privacy Leakage
The Ohio State University(俄亥俄州立大学);University of Chicago(芝加哥大学);University of Wisconsin, Madison(威斯康星大学麦迪逊分校);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);University of California, Los Angeles(加利福尼亚大学洛杉矶分校)
Grammar Based Speaker Role Identification for Air Traffic Control Speech Recognition
Idiap Research Institute(Idiap研究所);Brno University of Technology(布尔诺理工大学);Ecole Polytechnique Federale de Lausanne (EPFL)(洛桑联邦理工学院);University of Zurich(苏黎世大学);German Aerospace Center (DLR)(德国航空航天中心)
Discovering Phonetic Inventories with Crosslingual Automatic Speech Recognition
Center of Language and Speech Processing, The Johns Hopkins University(约翰斯·霍普金斯大学语言与语音处理中心);Human Language Technology Center of Excellence, The Johns Hopkins University(约翰斯·霍普金斯大学人类语言技术卓越中心);Multimedia Computing Group, Delft University of Technology(代尔夫特理工大学多媒体计算组);Department of Electrical and Computer Engineering, University of Illinois(伊利诺伊大学电气与计算机工程系)
Rethinking Speech Recognition with A Multimodal Perspective via Acoustic and Semantic Cooperative Decoding
School of Computer and Communication Engineering, University of Science and Technology Beijing(北京科技大学计算机与通信工程学院);USTB-EEasyTech Joint Lab of Artificial Intelligence, University of Science and Technology Beijing(北京科技大学USTB-易智瑞人工智能联合实验室);EEasy Technology Company Ltd.(易智科技有限公司)
Large Language Models based ASR Error Correction for Child Conversations
University of Southern California(美国南加州大学);Korea University(韩国大学);University of California, San Francisco(旧金山大学);University of California, Los Angeles(洛杉矶大学);Viterbi School of Engineering(工程学院);School of Psychology(心理学系);Weill Institute for Neurosciences(神经科学研究所);Semel Institute of Neuroscience and Human Behavior(神经科学与人类行为研究所)
Non-autoregressive Transformer with Unified Bidirectional Decoder for Automatic Speech Recognition
University of Science and Technology Beijing(北京科技大学);Key Laboratory of Knowledge Automation for Industrial Processes of Ministry of Education(教育部工业过程知识自动化重点实验室);EEasy Technology Company Ltd.(易捷科技有限公司)