级联等价假设：当语音大语言模型表现得像ASR→LLM流水线时
The Cascade Equivalence Hypothesis: When Do Speech LLMs Behave Like ASR$\rightarrow$LLM Pipelines?
AI总结：
本文研究语音大语言模型是否等价于ASR→LLM流水线，通过匹配骨干测试和机理分析，揭示其行为特性及在噪声下的性能差异。
摘要：
语音大语言模型被广泛认为比ASR→LLM级联更好，因为它们可以直接访问音频，而不是仅凭转录文本。在本文中，我们提出了一种评估方法，并对观察到的语音大语言模型行为进行了机理解释。首先，我们引入了匹配骨干测试，将语音大语言模型的行为与底层大语言模型的推理能力区分开来。其次，我们利用logit镜头和LEACE对语音大语言模型进行了机理分析，展示了从LLM隐藏状态中出现的字面转录文本以及文本表示的因果必要性。我们还表明，在大多数部署使用场景中，当前的语音大语言模型是昂贵的级联，且在噪声环境下表现更差，清洁条件下的优势可能在0dB时逆转高达7.6%。
英文摘要：
Speech LLMs are widely understood to be better than ASR$\rightarrow$LLM cascades since they have access to the audio directly, and not just the transcript. In this paper, we present an evaluation methodology and a mechanistic interpretation of the observed behavior of speech LLMs. First, we introduce matched-backbone testing which separates out the behavior of the speech LLM from the reasoning capabilities of the underlying LLM. Second, we provide a mechanistic analysis of speech LLMs using logit lens and LEACE and show the literal transcript emerging from the LLM's hidden states and that text representations are causally necessary. We also show that in most deployed use cases, current speech LLMs are expensive cascades, and under noise, they are worse ones, with clean-condition advantages reversing by up to 7.6% at 0dB.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
