发表机构
Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU); Universidad de Antioquia（埃尔朗根-纽伦堡弗里德里希-亚历山大大学; 安蒂奥基亚大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本研究提出将音频模型PhonoQ的结构化音系表征用于音频-发音rtMRI语音分类，可提升音系目标宏F1值与细粒度39音素分类性能，验证了音系信息的迁移价值。
AI 中文摘要
实时MRI可观测语音过程中的声道发音动作，但将这些发音模式映射到语音与音系类别仍具挑战性。本研究探究经训练以识别结构化音系特征的音频模型PhonoQ，是否可为音频-发音建模提供有用信息。具体而言，我们从PhonoQ的Conformer模块提取表征，该模块的训练受方式、部位、清浊及元音特征的监督。结合发音轮廓与同步音频衍生特征，我们对比WavLM-large与HuBERT-large基线模型，及纳入PhonoQ衍生表征的模型。在未见过的语音与未见过的受试者设置下，这些特征提升了音系目标的宏F1值，涵盖方式、部位、清浊、元音高度及元音后缩特征，还改善了细粒度的39音素分类。在仅用轮廓的推理设置中，音频衍生的教师监督相比仅轮廓训练产生适度但一致的提升，表明同步音频的音系信息可部分迁移至发音模型。最后，后验分析显示出可解释的表面敏感模式，与类似闪音的/t/实现、/t/-/r/后缩或塞擦化、鼻音部位同化一致。
英文摘要
Real-time MRI makes it possible to observe vocal-tract articulation during speech, but mapping these articulatory patterns to phonetic and phonological categories remains challenging. We investigate whether PhonoQ, an audio-based model trained to recognize structured phonological features, provides useful information for audio--articulatory modeling. Specifically, we extract representations from PhonoQ's Conformer module, whose training is shaped by supervision for manner, place, voicing, and vowel features. Using articulatory contours with synchronized audio-derived features, we compare WavLM-large and HuBERT-large baselines with models that incorporate PhonoQ-derived representations. Across unseen-speech and unseen-subject settings, these features improve macro-F1 for phonological targets including manner, place, voicing, vowel height, and vowel backness, and also improve fine-grained 39-phoneme classification. In a contour-only inference setting, audio-derived teacher supervision yields modest but consistent gains over contour-only training, indicating that phonological information from synchronized audio can be partially transferred to articulatory models. Finally, posterior analyses show interpretable surface-sensitive patterns consistent with flapping-like /t/ realizations, /t/-/r/ retraction or affrication, and nasal place assimilation.
CommentsSubmitted for review at SLT 2026
