Hearable devices, equipped with one or more microphones, are commonly used for speech communication. Here, we consider the scenario where a hearable is used to capture the user's own voice in a noisy environment. In this scenario, own voice reconstruction (OVR) is essential for enhancing the quality and intelligibility of the recorded noisy own voice signals. In previous work, we developed a deep learning-based OVR system, aiming to reduce the amount of device-specific recordings for training by using data augmentation with phoneme-dependent models of own voice transfer characteristics. Given the limited computational resources available on hearables, in this paper we propose low-complexity variants of an OVR system based on the FT-JNF architecture and investigate the required amount of device-specific recordings for effective data augmentation and fine-tuning. Simulation results show that the proposed OVR system considerably improves speech quality, even under constraints of low complexity and a limited amount of device-specific recordings.
Fraunhofer Institute for Digital Media Technology IDMT（弗劳恩霍夫数字媒体技术研究所（IDMT））；University of Oldenburg（奥尔登堡大学）
In-ear microphones in hearing protection devices can be utilized to capture the own voice speech of the person wearing the devices in noisy environments. Since in-ear recordings of the own voice are typically band-limited, an own voice reconstruction system is required to recover clean broadband speech from the in-ear signals. However, the availability of speech data for this scenario is typically limited due to device-specific transfer characteristics and the need to collect data from in-situ measurements. In this paper, we apply a deep learning-based bandwidth-extension system to the own voi…
In this paper we consider an in-ear headphone equipped with an inner microphone and multiple loudspeakers and we propose an optimization procedure with a convex objective function to derive a fixed multi-loudspeaker ANC controller aiming at minimizing the sound pressure at the ear drum. Based on the virtual microphone arrangement (VMA) technique and measured acoustic paths between the loudspeakers and the ear drum, the FIR filters of the ANC controller are jointly optimized to minimize the power spectral density at the ear drum, subject to design and stability constraints. For an in-ear headph…
Aristotle University of Thessaloniki（塞萨洛尼基亚里士多德大学）；Harokopio University of Athens（雅典哈罗科皮奥大学）
The importance of automated and objective monitoring of dietary behavior is becoming increasingly accepted. The advancements in sensor technology along with recent achievements in machine-learning--based signal-processing algorithms have enabled the development of dietary monitoring solutions that yield highly accurate results. A common bottleneck for developing and training machine learning algorithms is obtaining labeled data for training supervised algorithms, and in particular ground truth annotations. Manual ground truth annotation is laborious, cumbersome, can sometimes introduce errors,…
University of Oldenburg（奥尔登堡大学）；Jade Hochschule（Jade应用科学大学）
In this paper we consider an in-ear headphone equipped with an external microphone and aim to minimize the sound pressure at the ear drum by means of a fixed feedforward ANC controller. Based on measured acoustic paths to predict the sound pressure generated by external sources and the headphone at the ear drum, the FIR filter coefficients of the ANC controller are optimized for different sound fields. Due to the acoustic feedback path between the loudspeaker and the microphone, a stability constraint based on the Nyquist stability criterion is introduced. Performance degradations due to reins…
We present a database of acoustic transfer functions of the Hearpiece, an openly available multi-microphone multi-driver in-the-ear earpiece for hearing device research. The database includes HRTFs for 87 incidence directions as well as responses of the drivers, all measured at the four microphones of the Hearpiece as well as the eardrum in the occluded and open ear. The transfer functions were measured in both ears of 25 human subjects and a KEMAR with anthropometric pinnae for five reinsertions of the device. We describe the measurements of the database and analyse derived acoustic parameter…
Sound event localization and detection (SELD) is a combined task of identifying the sound event and its direction. Deep neural networks (DNNs) are utilized to associate them with the sound signals observed by a microphone array. Although ambisonic microphones are popular in the literature of SELD, they might limit the range of applications due to their predetermined geometry. Some applications (including those for pedestrians that perform SELD while walking) require a wearable microphone array whose geometry can be designed to suit the task. In this paper, for the development of such a wearabl…
Active noise cancellation on open-ear smart glasses
Department of Electrical and Computer Engineering, Carnegie Mellon University（卡内基梅隆大学电气与计算机工程系）；Department of Medical Physics and Acoustics, Carl von Ossietzky Universität Oldenburg（奥尔登堡大学医学物理与声学系）；College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）
WhisperMask: A Noise Suppressive Mask-Type Microphone for Whisper Speech
The University of Tokyo（东京大学）；National Institute of Advanced Industrial Science and Technology（独立行政法人产业技术综合研究所）；Sony Computer Science Laboratory（索尼计算机科学实验室）
Binaural audio gives the listener an immersive experience and can enhance augmented and virtual reality. However, recording binaural audio requires specialized setup with a dummy human head having microphones in left and right ears. Such a recording setup is difficult to build and setup, therefore mono audio has become the preferred choice in common devices. To obtain the same impact as binaural audio, recent efforts have been directed towards lifting mono audio to binaural audio conditioned on the visual input from the scene. Such approaches have not used an important cue for the task: the di…
