用于认知和物理负荷下语音分析的混合手工和可学习音频表示
Hybrid Handcrafted and Learnable Audio Representation for Analysis of Speech Under Cognitive and Physical Load
摘要：
作为对威胁或不利条件的一种神经生理反应，压力可影响认知、情绪和行为，在持续暴露的情况下对健康可能产生有害影响。由于言语的情感内容本质上是由个体的身心状态所调节的，因此大量的研究致力于研究压力诱导任务负荷的语言学相关性。历史上，语音重音分析（VSA）已经使用传统的数字信号处理（DSP）技术进行。尽管发展了基于深度神经网络（DNN）的现代方法，但由于各种各样的压力源和个体压力感知的相当大的变化，准确地检测语音中的压力仍然是困难的。为此，我们介绍了一组五个数据集的语音任务负载检测。这些录音是在志愿者群体中引起认知或身体压力时收集的，累计人数超过100人。我们使用这些数据集来设计和评估一种新的自监督音频表示，该表示利用了手工制作的功能（基于DSP）的有效性和数据驱动的DNN表示的复杂性。值得注意的是，所提出的方法优于广泛的手工特征集和新颖的基于DNN的音频表示学习方法。
英文摘要：
As a neurophysiological response to threat or adverse conditions, stress can affect cognition, emotion and behaviour with potentially detrimental effects on health in the case of sustained exposure. Since the affective content of speech is inherently modulated by an individual's physical and mental state, a substantial body of research has been devoted to the study of paralinguistic correlates of stress-inducing task load. Historically, voice stress analysis (VSA) has been conducted using conventional digital signal processing (DSP) techniques. Despite the development of modern methods based on deep neural networks (DNNs), accurately detecting stress in speech remains difficult due to the wide variety of stressors and considerable variability in the individual stress perception. To that end, we introduce a set of five datasets for task load detection in speech. The voice recordings were collected as either cognitive or physical stress was induced in the cohort of volunteers, with a cumulative number of more than a hundred speakers. We used the datasets to design and evaluate a novel self-supervised audio representation that leverages the effectiveness of handcrafted features (DSP-based) and the complexity of data-driven DNN representations. Notably, the proposed approach outperformed both extensive handcrafted feature sets and novel DNN-based audio representation learning approaches.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
