比房间更安静：语音编码器中的表示漂移与任务鲁棒性

中文摘要 AI 辅助

非语音干扰可以改变语音表示，而不会导致相应的任务损失。我们在四个任务上测试了八个冻结编码器，在整段录音、语音期间或停顿期间添加非语音声音。在整段录音干扰下，嵌入漂移与七种声音的任务损失相关，平均斯皮尔曼相关系数为0.81-0.88。将相同的声音在语音和停顿之间移动会改变这种模式。在安静到中等水平下，停顿干扰产生更大的漂移，而语音干扰通常在意图识别、说话人验证和语音识别上造成更大的损失。情感识别显示出较弱的放置效应。停顿干扰也会改变注入区域之外的语音帧表示。即使低于估计的录音背景，干扰也可以像重复语音片段一样改变嵌入。漂移有助于对不同声音的影响进行排序，但较大的漂移并不一致地表明更大的任务损失。

英文摘要

Non-speech interference can change a speech representation without causing comparable task loss. We test eight frozen encoders on four tasks, adding non-speech sounds throughout recordings, during speech, or in pauses. Under whole-recording interference, embedding drift tracks task loss across seven sounds, with mean Spearman correlations of 0.81-0.88. Moving the same sound between speech and pauses changes this pattern. At quiet to moderate levels, pause interference produces larger drift, while speech interference usually causes greater loss on intent recognition, speaker verification and speech recognition. Emotion recognition shows a weaker placement effect. Pause interference also changes speech-frame representations beyond the injected region. Even below the estimated recording background, interference can change embeddings as much as repeated speech takes do. Drift helps rank the effects of different sounds, but larger drift does not consistently indicate greater task loss.

发表机构

• Boston University（波士顿大学）
• Princeton University（普林斯顿大学）

机构由 AI 辅助整理，请以论文原文为准。