arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2406.09200cs.CL
收藏

自监督语音表征中说话人与语音信息的正交性和各向同性

Orthogonality and isotropy of speaker and phonetic information in self-supervised speech representations

University of Edinburgh(爱丁堡大学)

Mukhtar Mohamed, Oli Danyi Liu, Hao Tang, Sharon Goldwater

AI总结:

本文提出累积残差方差(CRV),用于衡量自监督语音表征中说话人与音素信息子空间的正交性及空间各向同性,并通过线性探测揭示其与音素识别准确率的关系。

摘要:

自监督语音表征能够极大地惠及下游语音技术,但其之所以有用的性质仍鲜为人知。已有研究假设,与表征空间几何结构相关的两个候选性质与下游任务具有良好相关性:(1)说话人质心和音素质心所张成子空间之间的正交程度;(2)空间的各向同性,即所有维度得到有效利用的程度。为研究这些性质,我们提出了一种新的度量——累积残差方差(Cumulative Residual Variance,CRV),可用于评估这两种性质。我们使用用于说话人识别和音素识别的线性分类器,探测六个不同自监督模型和两个未经训练基线的表征,并考察正交性或各向同性是否与线性探测准确率相关。我们发现,这两种度量都与语音探测准确率相关,不过关于各向同性的结果更为微妙。

英文摘要:

Self-supervised speech representations can hugely benefit downstream speech technologies, yet the properties that make them useful are still poorly understood. Two candidate properties related to the geometry of the representation space have been hypothesized to correlate well with downstream tasks: (1) the degree of orthogonality between the subspaces spanned by the speaker centroids and phone centroids, and (2) the isotropy of the space, i.e., the degree to which all dimensions are effectively utilized. To study them, we introduce a new measure, Cumulative Residual Variance (CRV), which can be used to assess both properties. Using linear classifiers for speaker and phone ID to probe the representations of six different self-supervised models and two untrained baselines, we ask whether either orthogonality or isotropy correlate with linear probing accuracy. We find that both measures correlate with phonetic probing accuracy, though our results on isotropy are more nuanced.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑