Evaluation of spatial audio reproduction schemes for application in hearing aid research
Universität Oldenburg（奥尔登堡大学）；HörTech gGmbH（HörTech有限责任公司）
Giso Grimm, Stephan Ewert, Volker Hohmann
备注：
The archived file is not the final published version of the article Evaluation of spatial audio reproduction schemes for application in hearing aid research, in Acta Acustica united with Acustica, volume 101, 2015, pp. 842-854(13). The definitive publisher-authenticated version is available online at http://www.ingentaconnect.com/content/dav/aaua. http://dx.doi.org/10.3813/AAA.918878
Loudspeaker-based spatial audio reproduction schemes are increasingly used for evaluating hearing aids in complex acoustic conditions. To further establish the feasibility of this approach, this study investigated the interaction between spatial resolution of different reproduction methods and technical and perceptual hearing aid performance measures using computer simulations. Three spatial audio reproduction methods -- discrete speakers, vector base amplitude panning and higher order ambisonics -- were compared in regular circular loudspeaker arrays with 4 to 72 channels. The influence of reproduction method and array size on performance measures of representative multi-microphone hearing aid algorithm classes with spatially distributed microphones and a representative single channel noise-reduction algorithm was analyzed. Algorithm classes differed in their way of analyzing and exploiting spatial properties of the sound field, requiring different accuracy of sound field reproduction. Performance measures included beam pattern analysis, signal-to-noise ratio analysis, perceptual localization prediction, and quality modeling. The results show performance differences and interaction effects between reproduction method and algorithm class that may be used for guidance when selecting the appropriate method and number of speakers for specific tasks in hearing aid research.
Current hearing aids normally provide amplification based on a general prescriptive fitting, and the benefits provided by the hearing aids vary among different listening environments despite the inclusion of noise suppression feature. Motivated by this fact, this paper proposes a data-driven machine learning technique to develop hearing aid fittings that are customised to speech in different noisy environments. A differentiable hearing loss model is proposed and used to optimise fittings with back-propagation. The customisation is reflected on the data of speech in different noise with also th…
Carl von Ossietzky Universität Oldenburg（奥尔登堡卡尔·冯·奥西茨基大学）；Hörzentrum Oldenburg gGmbH（奥尔登堡听力中心（非营利有限责任公司））
open Master Hearing Aid (openMHA) was developed and provided to the hearing aid research community as an open-source software platform with the aim to support sustainable and reproducible research towards improvement and new types of assistive hearing systems not limited by proprietary software. The software offers a flexible framework that allows the users to conduct hearing aid research using tools and a number of signal processing plugins provided with the software as well as the implementation of own methods. The openMHA software is independent of a specific hardware and supports Linux, ma…
Hearing aids are expected to improve speech intelligibility for listeners with hearing impairment. An appropriate amplification fitting tuned for the listener's hearing disability is critical for good performance. The developments of most prescriptive fittings are based on data collected in subjective listening experiments, which are usually expensive and time-consuming. In this paper, we explore an alternative approach to finding the optimal fitting by introducing a hearing aid speech processing framework, in which the fitting is optimised in an automated way using an intelligibility objectiv…
School of Biomedical Engineering, University of Sydney（悉尼大学生物医学工程学院）；School of Engg. & IT, UNSW Canberra（新南威尔士大学堪培拉分校工程与信息技术学院）；Curtin University（科廷大学）
Hearing aid is an electroacoustic device used to selectively amplify the audio sounds with an aim to make speech more intelligible for a hearing impaired person. Filter bank is one of the important parts of digital hearing aid where the sub band gains of each filter can be tuned to compensate an individuals unique hearing loss pattern. As the human perception is based on the logarithmic scale, nonuniform filter bank outperforms uniform filter bank. The main advantage of nonuniform filer bank is that it requires less number of sub-band filters, hence resulted in low hardware complexity and cost…
Research Center for Information Technology Innovation, Academia Sinica（中央研究院资讯科技创新研究中心）；Nagoya University（名古屋大学）；Institute of Information Science, Academia Sinica（中央研究院资讯科学研究所）；University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
Without the need of a clean reference, non-intrusive speech assessment methods have caught great attention for objective evaluations. Recently, deep neural network (DNN) models have been applied to build non-intrusive speech assessment approaches and confirmed to provide promising performance. However, most DNN-based approaches are designed for normal-hearing listeners without considering hearing-loss factors. In this study, we propose a DNN-based hearing aid speech assessment network (HASA-Net), formed by a bidirectional long short-term memory (BLSTM) model, to predict speech quality and inte…
University of Texas at Dallas（德克萨斯大学达拉斯分校）；Callier Center for Communication Disorders, University of Texas at Dallas（德克萨斯大学达拉斯分校卡利尔交流障碍中心）
Existing prescriptive compression strategies used in hearing aid fitting are designed based on gain averages from a group of users which are not necessarily optimal for a specific user. Nearly half of hearing aid users prefer settings that differ from the commonly prescribed settings. This paper presents a human-in-loop deep reinforcement learning approach that personalizes hearing aid compression to achieve improved hearing perception. The developed approach is designed to learn a specific user's hearing preferences in order to optimize compression based on the user's feedbacks. Both simulati…
Data Science Institute（数据科学研究所）；National University of Ireland Galway（爱尔兰国立高威大学）
Approximately 6.2% of the world's population (466 million people) suffer from disabling hearing impairment [1]. Hearing impairment impacts negatively on one's education, financial success [2][3], cognitive development in childhood [4], including increased risk of dementia in older adulthood [5]. Lack of or reduced social interaction due to hearing impairment affects creating or maintaining healthy relationships at home, school and work [5]. Hence, hearing impairment genuinely affects the overall quality of life and wellbeing. The cocktail party effect, which is a healthy hearing individual's a…
Reduction of unwanted environmental noises is an important feature of today's hearing aids (HA), which is why noise reduction is nowadays included in almost every commercially available device. The majority of these algorithms, however, is restricted to the reduction of stationary noises. In this work, we propose a denoising approach based on a three hidden layer fully connected deep learning network that aims to predict a Wiener filtering gain with an asymmetric input context, enabling real-time applications with high constraints on signal delay. The approach is employing a hearing instrument…
Audatic（Audatic公司）；Charité – Universitätsmedizin Berlin（柏林夏里特医学院）；Technische Universität Berlin（柏林工业大学）；Bernstein Center for Computational Neuroscience Berlin（柏林伯恩斯坦计算神经科学中心）
Almost half a billion people world-wide suffer from disabling hearing loss. While hearing aids can partially compensate for this, a large proportion of users struggle to understand speech in situations with background noise. Here, we present a deep learning-based algorithm that selectively suppresses noise while maintaining speech signals. The algorithm restores speech intelligibility for hearing aid users to the level of control subjects with normal hearing. It consists of a deep network that is trained on a large custom database of noisy speech signals and is further optimized by a neural ar…
In this letter, we derive a new super Gaussian Joint Maximum a Posteriori based single microphone speech enhancement gain function. The developed Speech Enhancement method is implemented on a smartphone, and this arrangement functions as an assistive device to hearing aids. We introduce a tradeoff parameter in the derived gain function that allows the smartphone user to customize their listening preference, by controlling the amount of noise suppression and speech distortion in real-time based on their level of hearing comfort perceived in noisy real world acoustic environment. Objective quali…
University of Edinburgh（爱丁堡大学）；Edinburgh Napier University（爱丁堡龙比亚大学）；University of Wolverhampton（伍尔弗汉普顿大学）
In this paper, we design a first of its kind transceiver (PHY layer) prototype for cloud-based audio-visual (AV) speech enhancement (SE) complying with high data rate and low latency requirements of future multimodal hearing assistive technology. The innovative design needs to meet multiple challenging constraints including up/down link communications, delay of transmission and signal processing, and real-time AV SE models processing. The transceiver includes device detection, frame detection, frequency offset estimation, and channel estimation capabilities. We develop both uplink (hearing aid…
University of Iowa（爱荷华大学）；University of Wisconsin-Madison（威斯康星大学麦迪逊分校）
Adults with mild-to-moderate hearing loss can use over-the-counter hearing aids to treat their hearing loss at a fraction of traditional hearing care costs. These products incorporate self-fitting methods that allow end-users to configure their hearing aids without the help of an audiologist. A self-fitting method helps users configure the gain-frequency responses that control the amplification for each frequency band of the incoming sound. This paper considers how to design effective self-fitting methods and whether we may evaluate certain aspects of their design without resorting to expensiv…
The proposed Circular statistics-based Inter-Microphone Phase difference estimation Localizer (CIMPL) method is tailored toward binaural hearing aid systems with microphone arrays in each unit. The method utilizes the circular statistics (circular mean and circular variance) of inter-microphone phase difference (IPD) across different microphone pairs. These IPDs are firstly mapped to time delays through a variance-weighted linear fit, then mapped to azimuth direction-of-arrival (DoA) and lastly information of different microphone pairs is combined. The variance is carried through the different…
DFingerNet: Noise-Adaptive Speech Enhancement for Hearing Aids
University of Augsburg（奥格斯堡大学）；Technical University of Munich（慕尼黑工业大学）；WS Audiology；Imperial College London（帝国理工学院）；Munich Data Science Institute（慕尼黑数据科学研究所）
University of Wolverhampton（伍尔弗汉普顿大学）；São Paulo State University（圣保罗州立大学）；Edinburgh Napier University（爱丁堡龙比亚大学）
Despite the recent success of machine learning algorithms, most models face drawbacks when considering more complex tasks requiring interaction between different sources, such as multimodal input data and logical time sequences. On the other hand, the biological brain is highly sharpened in this sense, empowered to automatically manage and integrate such streams of information. In this context, this work draws inspiration from recent discoveries in brain cortical circuits to propose a more biologically plausible self-supervised machine learning approach. This combines multimodal information us…
Edinburgh Napier University（爱丁堡龙比亚大学）；University of Wolverhampton（伍尔弗汉普顿大学）；University of Edinburgh（爱丁堡大学）
Multimodal hearing aids (HAs) aim to deliver more intelligible audio in noisy environments by contextually sensing and processing data in the form of not only audio but also visual information (e.g. lip reading). Machine learning techniques can play a pivotal role for the contextually processing of multimodal data. However, since the computational power of HA devices is low, therefore this data must be processed either on the edge or cloud which, in turn, poses privacy concerns for sensitive user data. Existing literature proposes several techniques for data encryption but their computational …
An accurate objective speech intelligibility prediction algorithms is of great interest for many applications such as speech enhancement for hearing aids. Most algorithms measures the signal-to-noise ratios or correlations between the acoustic features of clean reference signals and degraded signals. However, these hand-picked acoustic features are usually not explicitly correlated with recognition. Meanwhile, deep neural network (DNN) based automatic speech recogniser (ASR) is approaching human performance in some speech recognition tasks. This work leverages the hidden representations from D…
HAAQI-Net: A Non-intrusive Neural Music Audio Quality Assessment Model for Hearing Aids
Academia Sinica（中央研究院）；National Chengchi University（国立政治大学）；National Yang Ming Chiao Tung University（国立阳明交通大学）；Research Center for Information Technology Innovation, Academia Sinica（中央研究院资讯科技创新研究中心）；Institute of Information Science, Academia Sinica（中央研究院资讯科学研究所）；College of Informatics, National Chengchi University（国立政治大学资讯学院）；Institute of Communications Engineering, National Yang Ming Chiao Tung University（国立阳明交通大学通讯工程研究所）
In this paper, a binaural beamforming algorithm for hearing aid applications is introduced.The beamforming algorithm is designed to be robust to some error in the estimate of the target speaker direction. The algorithm has two main components: a robust target linearly constrained minimum variance (TLCMV) algorithm based on imposing two constraints around the estimated direction of the target signal, and a post-processor to help with the preservation of binaural cues. The robust TLCMV provides a good level of noise reduction and low level of target distortion under realistic conditions. The pos…
Many hearing-impaired listeners struggle to localize sounds due to poor availability of binaural cues. Listeners with a cochlear implant and a contralateral hearing aid -- so-called bimodal listeners -- are amongst the worst performers, as both interaural time and level differences are poorly transmitted. We present a new method to enhance head shadow in the low frequencies. Head shadow enhancement is achieved with a fixed beamformer with contralateral attenuation in each ear. The method results in interaural level differences which vary monotonically with angle. It also improves low-frequency…
University of Manchester（曼彻斯特大学）；University of Cambridge（剑桥大学）
Text-to-speech (TTS) systems offer the opportunity to compensate for a hearing loss at the source rather than correcting for it at the receiving end. This removes limitations such as time constraints for algorithms that amplify a sound in a hearing aid and can lead to higher speech quality. We propose an algorithm that restores loudness to normal perception at a high resolution in time, frequency and level, and embed it in a TTS system that uses Tacotron2 and WaveGlow to produce individually amplified speech. Subjective evaluations of speech quality showed that the proposed algorithm led to hi…
dCoNNear: An Artifact-Free Neural Network Architecture for Closed-loop Audio Signal Processing
Hearing Technology Lab, Department of Information Technology, Ghent University（信息科技系听力技术实验室，根特大学）；IDLAB, Department of Information Technology, Ghent University（信息科技系IDLAB，根特大学）
A Novel Uncertainty Parameter SR (Signal To Residual Spectrum Ratio) Evaluation Approach For Speech Enhancement
Gudlavalleru Engineering College（古德拉瓦莱鲁工程学院）
通常，听力受损的人使用助听器，其使用语音增强算法来实现。单通道语音增强系统中的语音估计和噪声估计是两个重要的组成部分。任何语音增强算法的主要目标都是在非平稳环境中估计噪声功率谱。VAD（语音活动检测器）用于识别语音停顿，并且在这些停顿期间仅估计噪声。MMSE（Minimum Mean Square Error）语音增强算法并没有增强语音的可懂度，语音质量和听者疲劳感是语音感知的方面。针对听障人群在非平稳环境中的失真控制问题，提出了一种基于不确定性参数的信号剩余谱比（SR）评价方法。该方法基于多阈值条件，将原始纯信号分为纯语音、准语音和非语音帧三部分，对噪声进行估计和更新。SR和LLR的不同值表明衰减和放大失真的量。该方法将与任何一种方法WAT（加权平均技术），因此，通过使用参数SR（信号与剩余频谱比）和LLR（对数像罩比），MMSE（最小均方误差）分段SNR和LLR。
This paper will describe a novel approach to the cocktail party problem that relies on a fully convolutional neural network (FCN) architecture. The FCN takes noisy audio data as input and performs nonlinear, filtering operations to produce clean audio data of the target speech at the output. Our method learns a model for one specific speaker, and is then able to extract that speakers voice from babble background noise. Results from experimentation indicate the ability to generalize to new speakers and robustness to new noise environments of varying signal-to-noise ratios. A potential applicati…
Eindhoven University of Technology（埃因霍温理工大学）；Nested Minds Solutions；GN Hearing（GN听力公司）
In this paper we present AIDA, which is an active inference-based agent that iteratively designs a personalized audio processing algorithm through situated interactions with a human client. The target application of AIDA is to propose on-the-spot the most interesting alternative values for the tuning parameters of a hearing aid (HA) algorithm, whenever a HA client is not satisfied with their HA performance. AIDA interprets searching for the "most interesting alternative" as an issue of optimal (acoustic) context-aware Bayesian trial design. In computational terms, AIDA is realized as an active…
Deep Learning for Personalized Binaural Audio Reproduction
Lab of Artificial Intelligence for Education, East China Normal University（人工智能教育实验室，华东师范大学）；Guangdong Key Laboratory of Intelligent Information Processing, College of Electronics and Information Engineering, Shenzhen University（智能信息处理重点实验室，深圳大学电子与信息工程学院）；Department of Computer Science and Technology, Tsinghua University（计算机科学与技术系，清华大学）；School of Electronics and Communication Engineering, Guangzhou University（电子与通信工程学院，广州大学）；School of Computer Science and Technology, East China Normal University（计算机科学与技术学院，华东师范大学）；Medizinische Physik, Carl von Ossietzky University of Oldenburg（医学物理，奥尔登堡大学）；Institute of Acoustics, Chinese Academy of Sciences（声学研究所，中国科学院）
University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
We consider the problem of audio voice separation for binaural applications, such as earphones and hearing aids. While today's neural networks perform remarkably well (separating $4+$ sources with 2 microphones) they assume a known or fixed maximum number of sources, K. Moreover, today's models are trained in a supervised manner, using training data synthesized from generic sources, environments, and human head shapes. This paper intends to relax both these constraints at the expense of a slight alteration in the problem definition. We observe that, when a received mixture contains too many so…
IRIT, Université de Toulouse, CNRS（图卢兹大学IRIT研究所，法国国家科学研究中心）；Tampere University（坦佩雷大学）
Audio source separation is usually achieved by estimating the short-time Fourier transform (STFT) magnitude of each source, and then applying a spectrogram inversion algorithm to retrieve time-domain signals. In particular, the multiple input spectrogram inversion (MISI) algorithm has been exploited successfully in several recent works. However, this algorithm suffers from two drawbacks, which we address in this paper. First, it has originally been introduced in a heuristic fashion: we propose here a rigorous optimization framework in which MISI is derived, thus proving the convergence of this…
Edinburgh Napier University（爱丁堡龙比亚大学）；University of Stirling（斯特灵大学）；University of Wolverhampton（伍尔弗汉普顿大学）
Noisy situations cause huge problems for suffers of hearing loss as hearing aids often make the signal more audible but do not always restore the intelligibility. In noisy settings, humans routinely exploit the audio-visual (AV) nature of the speech to selectively suppress the background noise and to focus on the target speaker. In this paper, we present a causal, language, noise and speaker independent AV deep neural network (DNN) architecture for speech enhancement (SE). The model exploits the noisy acoustic cues and noise robust visual cues to focus on the desired speaker and improve the sp…
