arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2407.13035cs.SDcs.CLcs.LGeess.AS
收藏

预训练基础模型表示揭示语音中的呼吸模式

Pre-Trained Foundation Model representations to uncover Breathing patterns in Speech

Apple(苹果公司)

Vikramjit Mitra, Anirban Chatterjee, Ke Zhai, Helen Weng, Ayuko Hill, Nicole Hay, Christopher Webb, Jamie Cheng, Erdrin Azemi

AI总结:

本研究提出基于预训练基础模型(Wav2Vec2)表示和Conv-LSTM的语音呼吸频率估计方法,在26人数据上实现低至1.6次/分钟的误差。

摘要:

人类语音产生过程涉及协调的呼吸动作以产生声学语音信号。通常,语音是在空气从肺部呼出并由声道调制时产生的,其间穿插着吸入空气(吸气)的时刻以再次充盈肺部。呼吸频率(RR)是评估个体整体健康、体能和一般健康状况的重要指标。现有的测量RR(每分钟呼吸次数)的方法需要使用专业设备或培训。研究表明,机器学习算法可以利用生物传感器信号作为输入来估计RR。基于语音的RR估计可以提供一种无需任何专业设备或传感器即可测量这一重要指标的有效方法。本研究探讨了一种基于机器学习的方法,从受试者对着近距离麦克风设备说话时获得的语音片段中估计RR。数据采集自N=26名个体,其中真实RR通过商用级胸带获取,并随后手动纠正任何错误。提出了一种卷积长短期记忆网络(Conv-LSTM)来从语音信号估计呼吸时间序列数据。我们证明,使用从基础模型(如Wav2Vec2)获得的预训练表示,与基线相比,可以以较低的均方根误差和较高的相关系数来估计呼吸时间序列。模型驱动的时间序列可用于估计RR,平均绝对误差(MAE)约为1.6次呼吸/分钟。

英文摘要:

The process of human speech production involves coordinated respiratory action to elicit acoustic speech signals. Typically, speech is produced when air is forced from the lungs and is modulated by the vocal tract, where such actions are interspersed by moments of breathing in air (inhalation) to refill the lungs again. Respiratory rate (RR) is a vital metric that is used to assess the overall health, fitness, and general well-being of an individual. Existing approaches to measure RR (number of breaths one takes in a minute) are performed using specialized equipment or training. Studies have demonstrated that machine learning algorithms can be used to estimate RR using bio-sensor signals as input. Speech-based estimation of RR can offer an effective approach to measure the vital metric without requiring any specialized equipment or sensors. This work investigates a machine learning based approach to estimate RR from speech segments obtained from subjects speaking to a close-talking microphone device. Data were collected from N=26 individuals, where the groundtruth RR was obtained through commercial grade chest-belts and then manually corrected for any errors. A convolutional long-short term memory network (Conv-LSTM) is proposed to estimate respiration time-series data from the speech signal. We demonstrate that the use of pre-trained representations obtained from a foundation model, such as Wav2Vec2, can be used to estimate respiration-time-series with low root-mean-squared error and high correlation coefficient, when compared with the baseline. The model-driven time series can be used to estimate $RR$ with a low mean absolute error (MAE) ~ 1.6 breaths/min.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑