arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2204.00465eess.AScs.AIeess.SP
收藏

用于发音表示分解的深度神经卷积矩阵分解

Deep Neural Convolutive Matrix Factorization for Articulatory Representation Decomposition

UC Berkeley(加州大学伯克利分校);Carnegie Mellon University(卡内基梅隆大学);University of Southern California(南加州大学)

Jiachen Lian, Alan W Black, Louis Goldstein, Gopala Krishna Anumanchipalli

摘要:

大多数关于数据驱动的语音表征学习的研究都集中在端到端的原始音频上,很少关注其内部的语音或手势结构。这项工作,调查来自发音运动学信号的语音表示,使用卷积稀疏矩阵分解的神经实现将发音数据分解成可解释的手势和手势分数。通过应用稀疏约束,手势分数利用语音手势的离散组合特性。此外,音素识别实验表明,手势分数确实成功地编码语音信息。因此,拟议的工作在发音音位学和深度神经网络之间架起了一座桥梁,以利用信息丰富、可理解、可解释和有效的语音表示。

英文摘要:

Most of the research on data-driven speech representation learning has focused on raw audios in an end-to-end manner, paying little attention to their internal phonological or gestural structure. This work, investigating the speech representations derived from articulatory kinematics signals, uses a neural implementation of convolutive sparse matrix factorization to decompose the articulatory data into interpretable gestures and gestural scores. By applying sparse constraints, the gestural scores leverage the discrete combinatorial properties of phonological gestures. Phoneme recognition experiments were additionally performed to show that gestural scores indeed code phonological information successfully. The proposed work thus makes a bridge between articulatory phonology and deep neural networks to leverage informative, intelligible, interpretable,and efficient speech representations.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑