This paper addresses the problem of automatic detection of voice pathologies directly from the speech signal. For this, we investigate the use of the glottal source estimation as a means to detect voice disorders. Three sets of features are proposed, depending on whether they are related to the speech or the glottal signal, or to prosody. The relevancy of these features is assessed through mutual information-based measures. This allows an intuitive interpretation in terms of discrimation power and redundancy between the features, independently of any subsequent classifier. It is discussed which characteristics are interestingly informative or complementary for detecting voice pathologies.
In most current approaches of speech processing, information is extracted from the magnitude spectrum. However recent perceptual studies have underlined the importance of the phase component. The goal of this paper is to investigate the potential of using phase-based features for automatically detecting voice disorders. It is shown that group delay functions are appropriate for characterizing irregularities in the phonation. Besides the respect of the mixed-phase model of speech is discussed. The proposed phase-based features are evaluated and compared to other parameters derived from the magn…
Brno University of Technology（布尔诺理工大学）；University of Las Palmas de Gran Canaria（大加那利岛拉斯帕尔马斯大学）
This paper describes a preliminary investigation of Voice Pathology Detection using Deep Neural Networks (DNN). We used voice recordings of sustained vowel /a/ produced at normal pitch from German corpus Saarbruecken Voice Database (SVD). This corpus contains voice recordings and electroglottograph signals of more than 2 000 speakers. The idea behind this experiment is the use of convolutional layers in combination with recurrent Long-Short-Term-Memory (LSTM) layers on raw audio signal. Each recording was split into 64 ms Hamming windowed segments with 30 ms overlap. Our trained model achieved…
GeHirNet: A Gender-Aware Hierarchical Model for Voice Pathology Classification
Centre for Digital Health Interventions（数字健康干预中心）；ETH Zurich（苏黎世联邦理工学院）；Institute of Mechanism Theory, Machine Dynamics and Robotics（机制理论、机械动力学与机器人研究所）；RWTH Aachen University（亚琛工业大学）；University of St. Gallen（圣加尔伦大学）
Faculty of Engineering and Technology（工程与技术学院）；Faculty of Audiology and Speech Language Pathology（听觉与语言病理学学院）；Sri Ramachandra Institute of Higher Education and Research（Sri Ramachandra 高等教育与研究学院）
Automatic objective non-invasive detection of pathological voice based on computerized analysis of acoustic signals can play an important role in early diagnosis, progression tracking and even effective treatment of pathological voices. In search towards such a robust voice pathology detection system we investigated 3 distinct classifiers within supervised learning and anomaly detection paradigms. We conducted a set of experiments using a variety of input data such as raw waveforms, spectrograms, mel-frequency cepstral coefficients (MFCC) and conventional acoustic (dysphonic) features (AF). In…
This paper presents a new method of singing voice analysis that performs mutually-dependent singing voice separation and vocal fundamental frequency (F0) estimation. Vocal F0 estimation is considered to become easier if singing voices can be separated from a music audio signal, and vocal F0 contours are useful for singing voice separation. This calls for an approach that improves the performance of each of these tasks by using the results of the other. The proposed method first performs robust principal component analysis (RPCA) for roughly extracting singing voices from a target music audio s…
Delft University of Technology（代尔夫特理工大学）；Universitat Politecnica de Catalunya（加泰罗尼亚理工大学）；University of Amsterdam（阿姆斯特丹大学）；Netherlands Cancer Institute（荷兰癌症研究所）；Johns Hopkins University（约翰斯·霍普金斯大学）
In this paper, we propose a new approach to pathological speech synthesis. Instead of using healthy speech as a source, we customise an existing pathological speech sample to a new speaker's voice characteristics. This approach alleviates the evaluation problem one normally has when converting typical speech to pathological speech, as in our approach, the voice conversion (VC) model does not need to be optimised for speech degradation but only for the speaker change. This change in the optimisation ensures that any degradation found in naturalness is due to the conversion process and not due t…
One-shot voice conversion (VC), which performs conversion across arbitrary speakers with only a single target-speaker utterance for reference, can be effectively achieved by speech representation disentanglement. Existing work generally ignores the correlation between different speech representations during training, which causes leakage of content information into the speaker representation and thus degrades VC performance. To alleviate this issue, we employ vector quantization (VQ) for content encoding and introduce mutual information (MI) as the correlation metric during training, to achiev…
Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；Ping An Technology (Shenzhen) Co., Ltd.（平安科技（深圳）有限公司）；The Chinese University of Hong Kong（香港中文大学）
One-shot voice conversion (VC) with only a single target speaker's speech for reference has become a hot research topic. Existing works generally disentangle timbre, while information about pitch, rhythm and content is still mixed together. To perform one-shot VC effectively with further disentangling these speech components, we employ random resampling for pitch and content encoder and use the variational contrastive log-ratio upper bound of mutual information and gradient reversal layer based adversarial mutual information learning to ensure the different parts of the latent space containing…
Building a high-quality singing corpus for a person who is not good at singing is non-trivial, thus making it challenging to create a singing voice synthesizer for this person. Learn2Sing is dedicated to synthesizing the singing voice of a speaker without his or her singing data by learning from data recorded by others, i.e., the singing teacher. Inspired by the fact that pitch is the key style factor to distinguish singing from speaking voice, the proposed Learn2Sing 2.0 first generates the preliminary acoustic feature with averaged pitch value in the phone level, which allows the training of…
University of Mons（蒙斯大学）；Trinity College Dublin（都柏林圣三一学院）
This paper investigates the temporal excitation patterns of creaky voice. Creaky voice is a voice quality frequently used as a phrase-boundary marker, but also as a means of portraying attitude, affective states and even social status. Consequently, the automatic detection and modelling of creaky voice may have implications for speech technology applications. The acoustic characteristics of creaky voice are, however, rather distinct from modal phonation. Further, several acoustic patterns can bring about the perception of creaky voice, thereby complicating the strategies used for its automatic…
A new algorithm for voice automatic syllabic splitting in the Portuguese language is proposed, which is based on the envelope of the speech signal of the input audio file. A computational implementation in MatlabTM is presented and made available at the URL http://www2.ee.ufpe.br/codec/divisao_silabica.html. Due to its straightforwardness, the proposed method is very attractive for embedded systems (e.g. i-phones). It can also be used as a screen to assist more sophisticated methods. Voice excerpts containing more than one syllable and identified by the same envelope are named as super-syllabl…
We propose a novel application based on acoustic-to-articulatory inversion towards quality assessment of voice converted speech. The ability of humans to speak effortlessly requires coordinated movements of various articulators, muscles, etc. This effortless movement contributes towards naturalness, intelligibility and speakers identity which is partially present in voice converted speech. Hence, during voice conversion, the information related to speech production is lost. In this paper, this loss is quantified for male voice, by showing increase in RMSE error for voice converted speech follo…
Controllable Dysarthric Speech Synthesis with Patient-Specific Conditioning for Speaker-Diverse ASR Augmentation
Department of Language Science and Technology, The Hong Kong Polytechnic University（语言科学与技术系，香港理工大学）；College of Rehabilitation Medicine, Fujian University of Traditional Chinese Medicine（康复医学学院，福建中医药大学）；Department of Imaging, Rehabilitation Hospital affiliated to Fujian University of Traditional Chinese Medicine（影像科，福建中医药大学附属康复医院）
Voice Activity Detection (VAD) refers to the problem of distinguishing speech segments from background noise. Numerous approaches have been proposed for this purpose. Some are based on features derived from the power spectral density, others exploit the periodicity of the signal. The goal of this paper is to investigate the joint use of source and filter-based features. Interestingly, a mutual information-based assessment shows superior discrimination power for the source-related features, especially the proposed ones. The features are further the input of an artificial neural network-based cl…
College of Geophysics, Chengdu University of Technology（成都理工大学地球物理学院）；Changhong AI Lab (CHAIR), Sichuan Changhong Electronics Holding Group Co., Ltd.（四川长虹电子控股集团有限公司长虹人工智能实验室）
Suffering from limited singing voice corpus, existing singing voice synthesis (SVS) methods that build encoder-decoder neural networks to directly generate spectrogram could lead to out-of-tune issues during the inference phase. To attenuate these issues, this paper presents a novel acoustic model with independent pitch encoder and phoneme encoder, which disentangles the phoneme and pitch information from music score to fully utilize the corpus. Specifically, according to equal temperament theory, the pitch encoder is constrained by a pitch metric loss that maps distances between adjacent inpu…
Vevo2: A Unified and Controllable Framework for Speech and Singing Voice Generation
The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；ByteDance Seed（字节跳动种子）；Shenzhen Loop Area Institute（深圳河套学院）；City University of Macau（澳门城市大学）；Amphion Technology Co., Ltd.（Amphion科技有限公司）
University of Mons（蒙斯大学）；Faculty of Engineering, University of Mons（蒙斯大学工程学院）
This paper addresses the issue of cough detection using only audio recordings, with the ultimate goal of quantifying and qualifying the degree of pathology for patients suffering from respiratory diseases, notably mucoviscidosis. A large set of audio features describing various aspects of the audio signal is proposed. These features are assessed in two steps. First, their intrisic potential and redundancy are evaluated using mutual information-based measures. Secondly, their efficiency is confirmed relying on three classifiers: Artificial Neural Network, Gaussian Mixture Model and Support Vect…
Learning Disentangled Speech Representations with Contrastive Learning and Time-Invariant Retrieval
Ping An Technology (Shenzhen) Co., Ltd.（平安科技（深圳）有限公司）；University of Science and Technology of China（中国科学技术大学）；Huya Inc (Shenzhen) Co., Ltd.（虎牙信息科技（深圳）有限公司）
SpeechFlow is a powerful factorization model based on information bottleneck (IB), and its effectiveness has been reported by several studies. A potential problem of SpeechFlow, however, is that if the IB channels are not well designed, the resultant factors cannot be well disentangled. In this study, we propose a CycleFlow model that combines random factor substitution and cycle loss to solve this problem. Experiments on voice conversion tasks demonstrate that this simple technique can effectively reduce mutual information among individual factors, and produce clearly better conversion than t…
Influence of Utterance and Speaker Characteristics on the Classification of Children with Cleft Lip and Palate
Technische Hochschule Nürnberg Georg Simon Ohm（纽伦堡乔治·西蒙·欧姆应用技术大学）；Friedrich-Alexander-Universität Erlangen-Nürnberg（埃尔朗根-纽伦堡弗里德里希-亚历山大大学）；Intel Labs（英特尔研究院）
Multi-class Detection of Pathological Speech with Latent Features: How does it perform on unseen data?
Technische Hochschule Nürnberg Georg Simon Ohm（纽伦堡乔治·西蒙·欧姆应用科学大学）；Friedrich-Alexander-University Erlangen-Nürnberg（埃尔朗根-纽伦堡弗里德里希-亚历山大大学）；Intel Labs（英特尔研究院）
本研究利用预训练 wav2vec 2.0 的潜在表示，结合神经网络、大间隔分类器和树提升机区分四种病理语音，在噪声和未见数据下展现出强鲁棒性与高 F1 分数。
Speaker-Disentangled Remote Speech Detection of Asthma and COPD Exacerbations
Institute of Data Science, Maastricht University（数据科学研究所，马斯特里赫特大学）；NUTRIM Research Institute of Nutrition and Translational Research in Metabolism, Faculty of Health Medicine and Life Sciences, Maastricht University（营养与代谢转化研究营养研究所，健康医学与生命科学学院，马斯特里赫特大学）；Maastricht University Medical Centre（马斯特里赫特大学医学中心）
University of Mons（蒙斯大学）；Cliniques St Luc（圣卢克诊所）；University of Louvain（鲁汶大学）
In the framework of assessing the pathology severity in chronic cough diseases, medical literature underlines the lack of tools for allowing the automatic, objective and reliable detection of cough events. This paper describes a system based on two microphones which we developed for this purpose. The proposed approach relies on a large variety of audio descriptors, an efficient algorithm of feature selection based on their mutual information and the use of artificial neural networks. First, the possible use of a contact microphone (placed on the patient's thorax or trachea) in complement to th…
Center for Robust Speech Systems, The University of Texas at Dallas（德克萨斯大学达拉斯分校鲁棒语音系统中心）
Despite speaker verification has achieved significant performance improvement with the development of deep neural networks, domain mismatch is still a challenging problem in this field. In this study, we propose a novel framework to disentangle speaker-related and domain-specific features and apply domain adaptation on the speaker-related feature space solely. Instead of performing domain adaptation directly on the feature space where domain information is not removed, using disentanglement can efficiently boost adaptation performance. To be specific, our model's input speech from the source a…
