arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.03096eess.AScs.CL
收藏

解读自监督语音特征的维度中说话人特性

Interpreting Speaker Characteristics in the Dimensions of Self-Supervised Speech Features

Department of Electrical and Electronic Engineering, Stellenbosch University(斯特伦博斯大学电气与电子工程系);Concordia University(康科迪亚大学)

Kyle Janse van Rensburg, Benjamin van Niekerk, Herman Kamper

AI总结:

本文通过PCA分析自监督语音特征的维度,揭示了语音特征如音高、性别等在主成分中的分布及相互影响,展示了特征维度的独立性和可操控性。

摘要:

语音模型通过自监督学习训练如何构建其表示?先前研究关注不同层中信息如何编码在特征向量中。但很少有研究考虑语音特性是否在SSL特征的个体维度中被捕捉。本文通过PCA分析 utterance-averaged 表示,针对多种SSL模型发现,解释最大方差的主维度编码音高及相关特性如性别。其他个体主维度与强度、噪声水平、第二共振峰及高频特性相关。我们进一步通过合成分析表明,大多数特征维度彼此影响较小。此外,特征可通过操控相应维度进行改变。

英文摘要:

How do speech models trained through self-supervised learning structure their representations? Previous studies have looked at how information is encoded in feature vectors across different layers. But few studies have considered whether speech characteristics are captured \textit{within} individual dimensions of SSL features. In this paper, we specifically look at speaker information using PCA on utterance-averaged representations. For a range of SSL models, we find that the principal dimension that explains most variance encodes pitch and associated characteristics like gender. Other individual principal dimensions correlate with intensity, noise levels, the second formant, and higher frequency characteristics. We then use synthesis analyses to show that the dimensions for most characteristics are isolated from each other's influence. We further show that characteristics can be changed by manipulating the corresponding dimensions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑