现代语音增强算法通过使用大型递归神经网络(RNN)实现了显著的噪声抑制。然而,大型RNN限制了助听器硬件(HW)形状因子的实际部署,这些硬件是电池供电的,并且在具有有限存储容量和计算能力的资源受限的微控制器单元(MCU)上运行。在这项工作中,我们使用模型压缩技术来弥合这一差距。我们定义了HW对RNN施加的约束,并描述了满足这些约束的方法。尽管模型压缩技术是一个活跃的研究领域,但我们是第一个使用权重/激活的修剪和整数量化来证明其对RNN语音增强的功效的。我们还演示了状态更新跳过,这减少了计算负载。最后,我们对压缩模型进行感知评估,以验证人类评分员的音频质量。结果显示,与压缩模型的基线相比,模型大小和操作分别减少了11.9 $\times$和2.9 $\times$,在收听偏好方面没有统计学差异,仅表现出0.55 dB SDR的损失。我们的模型实现了2.39毫秒的计算延迟,以及在10毫秒的目标和351 $\times$比以前的工作更好。
英文摘要:
Modern speech enhancement algorithms achieve remarkable noise suppression by means of large recurrent neural networks (RNNs). However, large RNNs limit practical deployment in hearing aid hardware (HW) form-factors, which are battery powered and run on resource-constrained microcontroller units (MCUs) with limited memory capacity and compute capability. In this work, we use model compression techniques to bridge this gap. We define the constraints imposed on the RNN by the HW and describe a method to satisfy them. Although model compression techniques are an active area of research, we are the first to demonstrate their efficacy for RNN speech enhancement, using pruning and integer quantization of weights/activations. We also demonstrate state update skipping, which reduces the computational load. Finally, we conduct a perceptual evaluation of the compressed models to verify audio quality on human raters. Results show a reduction in model size and operations of 11.9$\times$ and 2.9$\times$, respectively, over the baseline for compressed models, without a statistical difference in listening preference and only exhibiting a loss of 0.55dB SDR. Our model achieves a computational latency of 2.39ms, well within the 10ms target and 351$\times$ better than previous work.
DFingerNet: Noise-Adaptive Speech Enhancement for Hearing Aids
University of Augsburg(奥格斯堡大学);Technical University of Munich(慕尼黑工业大学);WS Audiology;Imperial College London(帝国理工学院);Munich Data Science Institute(慕尼黑数据科学研究所)
Overview Of The 2023 Icassp Sp Clarity Challenge: Speech Enhancement For Hearing Aids
Acoustics Research Centre, University of Salford(索尔福德大学声学研究中心);Department of Computer Science, University of Sheffield(谢菲尔德大学计算机科学系);School of Medicine, University of Nottingham(诺丁汉大学医学院);School of Psychology, Cardiff University(卡迪夫大学心理学院)
Feasibility of Time-Domain DNN-Based Speech Enhancement on Embedded FPGA for Hearing Aids
Radboud University, Donders Institute for Brain, Cognition, and Behaviour, The Netherlands(拉德堡德大学,脑认知行为研究所,荷兰);Mortimer B. Zuckerman Mind, Brain, Behavior Institute, Columbia University, USA(莫蒂默·B·齐克曼心智、大脑与行为研究所,哥伦比亚大学,美国)
ABSE-NET: A Lightweight Neural Model for Active Binaural Speech Enhancement in Open-Fit Hearing Aids
College of Computer Science, Inner Mongolia University(内蒙古大学计算机学院);College of Electronic and Information Engineering, Inner Mongolia University(内蒙古大学电子与信息工程学院)
Evaluating the Intelligibility Benefits of Neural Speech Enrichment for Listeners with Normal Hearing and Hearing Impairment using the Greek Harvard Corpus
University of Crete(克里特大学);Chania General Hospital(干尼亚综合医院)
Restoring speech intelligibility for hearing aid users with deep learning
Audatic(Audatic公司);Charité – Universitätsmedizin Berlin(柏林夏里特医学院);Technische Universität Berlin(柏林工业大学);Bernstein Center for Computational Neuroscience Berlin(柏林伯恩斯坦计算神经科学中心)
Head shadow enhancement with low-frequency beamforming improves sound localization and speech perception for simulated bimodal listeners
KU Leuven – University of Leuven(鲁汶大学)
由于双耳线索的可用性差,许多听力受损的听众难以定位声音。带有人工耳蜗和对侧助听器的听众--所谓的双峰听众--是表现最差的,因为两耳间的时间和水平差都很难传递。提出了一种新的低频头部阴影增强方法。头部阴影增强是通过在每只耳朵中使用对侧衰减的固定波束形成器来实现的。该方法导致耳间水平差随角度单调变化。在语音和噪声空间分离的情况下,它还提高了低频信噪比。我们验证了该方法在两个实验中的声学模拟双峰听。在定位实验中,与标准全向麦克风相比,性能从50.5°提高到26.8°均方根误差。在噪声中的语音实验中,语音从正面方向呈现。当噪声来自人工耳蜗侧时,语音接收阈值提高了15.7 dB SNR,当噪声来自助听器侧时,语音接收阈值提高了7.6 dB SNR,当噪声来自各个方向时,语音接收阈值不受影响。除了双模态听众,该方法也可能是有前途的双侧人工耳蜗或助听器用户。其低计算复杂度使得该方法适合于在当前临床设备中的应用。保留字:头影增强、耳间水平差增强、声音定位、定向听力、噪声中言语、言语清晰度PACS:43.60.Fg、43.66.Pn、43.66.Qp、43.66.Rq、43.66.Ts、43.71.- k,43.71.Es,43.71.Ky
A Novel Uncertainty Parameter SR (Signal To Residual Spectrum Ratio) Evaluation Approach For Speech Enhancement
Gudlavalleru Engineering College(古德拉瓦莱鲁工程学院)
通常,听力受损的人使用助听器,其使用语音增强算法来实现。单通道语音增强系统中的语音估计和噪声估计是两个重要的组成部分。任何语音增强算法的主要目标都是在非平稳环境中估计噪声功率谱。VAD(语音活动检测器)用于识别语音停顿,并且在这些停顿期间仅估计噪声。MMSE(Minimum Mean Square Error)语音增强算法并没有增强语音的可懂度,语音质量和听者疲劳感是语音感知的方面。针对听障人群在非平稳环境中的失真控制问题,提出了一种基于不确定性参数的信号剩余谱比(SR)评价方法。该方法基于多阈值条件,将原始纯信号分为纯语音、准语音和非语音帧三部分,对噪声进行估计和更新。SR和LLR的不同值表明衰减和放大失真的量。该方法将与任何一种方法WAT(加权平均技术),因此,通过使用参数SR(信号与剩余频谱比)和LLR(对数像罩比),MMSE(最小均方误差)分段SNR和LLR。
Multi-Speaker DOA Estimation in Binaural Hearing Aids using Deep Learning and Speaker Count Fusion
School of Electrical Engineering and Computer Science, University of Ottawa, Canada(Ottawa大学电气与计算机科学学院, 加拿大);Department of Electrical and Computer Engineering, Universite de Sherbrooke, Canada(Sherbrooke大学电气与计算机工程系, 加拿大);WS Audiology, Germany(德国WS听觉学)
Audiovisual joint learning for end-to-end hearing aids
Graduate Institute of Communication Engineering, National Taiwan University(国立台湾大学通讯工程学研究所);Research Center for Information Technology Innovation, Academia Sinica(中央研究院资讯科技创新研究中心);Department of Otolaryngology-Head and Neck Surgery, Taipei Veterans General Hospital(台北荣民总医院耳鼻喉头颈外科);School of Medicine, National Yang Ming Chiao Tung University(国立阳明交通大学医学院);Center for Hearing Research, University of California, Irvine(加州大学欧文分校听力研究中心)
Advances in Intelligent Hearing Aids: Deep Learning Approaches to Selective Noise Cancellation
Department of Electrical Engineering, National University of Sciences and Technology (NUST), Center for Excellence in Manufacturing Engineering (CEME)(电子工程系,国立科学与技术大学(NUST),制造工程卓越中心)