arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~

学习目标调控感知窄化:自监督语音编码器的跨语言、分层、十种子研究

The Learning Objective Governs Perceptual Narrowing: A Cross-Lingual, Layer-Wise, Ten-Seed Study of Self-Supervised Speech Encoders

Sejin Yoo

arXiv 2608.00507首次发表:更新:

AI 中文总结

该研究通过十种子实验发现,自监督语音编码器的学习目标(而非架构)是感知窄化型表征变化的首要决定因素,重构与预测目标对非母语音素辨别能力的影响相反。

AI 中文摘要

感知窄化是生命第一年出现的非母语音素辨别能力发育丧失的典型发育现象,但哪种学习目标会产生该现象仍不明确。我们在儿童导向语音和朗读语音上训练了一个约700万参数的Transformer编码器,以种子为复制单元,在英语、法语和汉语上进行十种子的音素ABX评估,共得到六项结果:一、学习目标决定跨语言迁移的方向:重构(掩码梅尔预测)会降低非母语辨别能力,预测(帧对比)会提升该能力——在相同编码器、相同数据的情况下,第一层汉语ABX得分存在+0.051的差距(p=3×10^-8),20次运行的符号完全一致;二、该下降结合了较大的固有难度梯度和较小的语言专业化效应(匹配与不匹配条件的差距为+0.022,p=10^-4,覆盖全部四层);三、在语言对称的原始梅尔基准下,重构会使第一层的可辨别性低于输入,预测则使其高于输入;四、朗读语音的非母语下降幅度是儿童导向语音的3.6倍;五、常规的三种子预算无法可靠观测到该效应:十种子下明确的效应,仅70%的三种子子集就判定为显著;六、六种目标配置(锐化、压缩、巩固、它们的组合及两种形式的词级语义接地)均无法产生完整的发育特征(母语提升且非母语下降):单一目标会以相同方向作用于两种语言,因为它作用于共享表征。我们得出结论:是学习目标而非架构,是窄化型表征变化的首要决定因素。

英文摘要

Perceptual narrowing---the developmental loss of non-native phoneme discrimination in the first year of life \citep{werker1984}---is a canonical developmental finding, yet \emph{what learning objective produces it} remains open. We train a \(\sim\)7\,M-parameter Transformer encoder on child-directed and read speech and evaluate phoneme ABX in English, French, and Mandarin over ten seeds, the seed as the unit of replication. Six results. \textbf{(1)}~The objective sets the direction of cross-lingual transfer: reconstruction (masked mel-prediction) degrades non-native discrimination, prediction (frame-contrastive) improves it---a same-encoder, same-data gap of \(+0.051\) in first-layer Mandarin ABX (\(p=3\times10^{-8}\)), unanimous in sign across twenty runs. \textbf{(2)}~That decline combines a large arm-intrinsic difficulty gradient with a smaller language-specialization effect (matched vs.\ mismatched \(+0.022\), \(p=10^{-4}\), all four layers). \textbf{(3)}~Against a language-symmetric raw-mel floor, reconstruction pushes the first layer \emph{below} the discriminability of its input; prediction pushes it \emph{above}. \textbf{(4)}~Read speech gives a \(3.6\times\) steeper non-native decline than child-directed speech. \textbf{(5)}~The customary three-seed budget cannot see this reliably: an effect unambiguous at ten seeds is called significant by as few as 70\% of three-seed subsets. \textbf{(6)}~Six objective configurations---sharpening, compression, consolidation, their composition, and word-level semantic grounding in two forms---fail to produce the full developmental signature (native improves \emph{and} non-native declines): a single objective moves both languages the same way because it acts on a shared representation. We conclude that the objective, not the architecture, is the first-order determinant of narrowing-shaped representational change.

Comments11 pages, 6 figures

论文原文

arXiv 摘要页 · PDF 原文 · HTML 原文

↑