发表机构
University of Southern California（南加州大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文利用稀疏自编码器引导神经音频编解码器中的年龄、性别和口音信息，发现所选维度可引发目标偏移，但存在质量下降，凸显了分离特质信息的必要性。
AI 中文摘要
神经音频编解码器（NACs）广泛应用于语音生成和音频语言建模，然而它们如何编码说话者特质信息仍鲜为人知。先前的工作应用稀疏自编码器（SAEs）通过任务级分析来研究NACs中的口音信息。在此，我们将该分析扩展到波形级别，并扩展到年龄、性别和口音，使用SAE引导来探测稀疏激活中的特质相关信息。我们识别与特质相关的维度，修改其激活，并评估由此重建的语音。在五种NACs中，引导所选维度会在说话者特质预测中引发目标导向的偏移。在Mimi上的随机维度基线产生的偏移较小，支持所选维度的相关性。然而，响应因编解码器、特质和引导方向而异，且增加引导强度并不一致地放大预期偏移。引导通常还会增加词错误率并降低预测的感知质量。这些发现表明，SAEs在可引导的激活中捕获说话者特质信息，而伴随的质量下降凸显了更好地区分特质相关信息与其他信息的必要性。
英文摘要
Neural audio codecs (NACs) are widely used in speech generation and audio-language modeling, yet how they encode speaker-trait information remains poorly understood. Prior work applied sparse autoencoders (SAEs) to investigate accent information in NACs through task-level analysis. Here, we extend this analysis to the waveform level and to age, gender, and accent, using SAE steering to probe trait-related information in sparse activations. We identify trait-associated dimensions, modify their activations, and evaluate the resulting reconstructed speech. Across five NACs, steering the selected dimensions induces target-directed shifts in speaker-trait predictions. A random-dimension baseline on Mimi produces smaller shifts, supporting the relevance of the selected dimensions. However, responses vary across codecs, traits, and steering directions, and increasing steering strength does not consistently amplify the intended shifts. Steering also generally increases word error rates and lowers predicted perceptual quality. These findings suggest that SAEs capture speaker-trait information in steerable activations, while the accompanying quality degradation highlights the need to better separate trait-related information from other information.
