对话蒸馏：用于基于LLM的ASR的对话音频上下文抽象压缩
Distilling Conversations: Abstract Compression of Conversational Audio Context for LLM-based ASR
AI总结：
本文研究了多模态上下文对基于LLM的ASR的提升效果及高效表示方法，提出抽象压缩技术以减少先前对话的音频负载，实验证明其在领域内和领域外数据集上均有效。
摘要：
标准的基于LLM的语音识别系统通常孤立处理语句，限制了其利用对话上下文的能力。本文研究了先前轮次的多模态上下文是否能提升基于LLM的ASR，并探讨如何高效表示该上下文。我们发现，在监督的多轮训练后，对话上下文主要有助于上下文实体的识别。然而，条件于原始上下文是昂贵的，因为先前轮次的音频标记序列随着对话长度迅速增长。为了解决这个问题，我们提出了抽象压缩，将先前轮次的音频部分替换为固定数量的学得的潜在标记，同时保留对应的转录文本。在领域内和领域外的测试集上，压缩模型在较小的先前轮次音频负载下恢复了部分原始上下文条件的收益。我们还提供了压缩设置的针对性分析及其权衡。
英文摘要：
Standard LLM-based speech recognition systems typically process utterances in isolation, limiting their ability to leverage conversational context. In this work, we study whether multimodal context from prior turns improves LLM-based ASR and how to represent that context efficiently. We find that, after supervised multi-turn training, conversational context mainly helps with the recognition of contextual entities. However, conditioning on raw context is expensive because the prior-turn audio token sequence grows rapidly with conversation length. To address this, we propose Abstract Compression, which replaces the audio portion of prior turns with a fixed number of learned latent tokens while retaining corresponding transcripts explicitly. On both in-domain and out-of-domain test sets, the compressed model recovers part of the gains of raw-context conditioning with a smaller prior-turn audio footprint. We also provide targeted analyses of the compression setup and its trade-offs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
