arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2407.16417cs.SDcs.LGeess.AS
收藏

语音与音频基础模型在狨猴叫声分析中的效用

On the Utility of Speech and Audio Foundation Models for Marmoset Call Analysis

Idiap Research Institute(伊迪亚普研究所);Ecole polytechnique fédérale de Lausanne(洛桑联邦理工学院)

Eklavya Sarkar, Mathew Magimai. -Doss

AI总结:

本研究评估语音与音频基础模型在狨猴叫声类型与叫声者分类任务中的效用,发现更高预训练带宽能提升性能,且语音与通用音频预训练结果相当,均优于频谱基线。

摘要:

狨猴猴在其叫声中编码了至关重要的信息,并作为神经生物学家理解人类发声交流进化起源的替代模型。传统上使用基于信号处理的特征进行分析,最近的方法利用在人类语音上预训练的自监督模型进行特征提取,利用其独立于声学域学习信号内在结构的能力。然而,此类基础模型在多类分类、带宽和预训练域方面对于狨猴叫声分析的效用仍不清楚。本研究评估了从语音和通用音频域派生的特征表示,在4、8和16 kHz的预训练带宽下,用于狨猴叫声类型和叫声者分类任务。结果表明,具有更高带宽的模型能提高性能,并且在语音或通用音频上进行预训练可产生可比的结果,优于频谱基线。

英文摘要:

Marmoset monkeys encode vital information in their calls and serve as a surrogate model for neuro-biologists to understand the evolutionary origins of human vocal communication. Traditionally analyzed with signal processing-based features, recent approaches have utilized self-supervised models pre-trained on human speech for feature extraction, capitalizing on their ability to learn a signal's intrinsic structure independently of its acoustic domain. However, the utility of such foundation models remains unclear for marmoset call analysis in terms of multi-class classification, bandwidth, and pre-training domain. This study assesses feature representations derived from speech and general audio domains, across pre-training bandwidths of 4, 8, and 16 kHz for marmoset call-type and caller classification tasks. Results show that models with higher bandwidth improve performance, and pre-training on speech or general audio yields comparable results, improving over a spectral baseline.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑