Seal：推进语音语言模型成为少样本学习者
Seal: Advancing Speech Language Models to be Few-Shot Learners
AI总结：
本文提出Seal模型，通过KL散度损失训练投影器连接冻结的语音编码器和语言模型解码器，使语音语言模型具备少样本学习能力，并在两个语音理解任务上验证了其稳健性。
摘要：
现有的自回归语言模型已经展示出通过提示中的几个示例即可执行新任务的卓越能力，而无需任何额外训练。为了将这种能力扩展到多模态（即语音和语言）设置，本文引入了Seal模型，即语音语言模型的缩写。它采用了一种新颖的对齐方法，其中执行Kullback-Leibler散度损失来训练一个投影器，该投影器将冻结的语音编码器与冻结的语言模型解码器连接起来。由此产生的Seal模型在两个语音理解任务上展现出作为少样本学习者的稳健性能。此外，还进行了一致性实验，以验证其在不同预训练语言模型上的稳健性。
英文摘要：
Existing auto-regressive language models have demonstrated a remarkable capability to perform a new task with just a few examples in prompt, without requiring any additional training. In order to extend this capability to a multi-modal setting (i.e. speech and language), this paper introduces the Seal model, an abbreviation for speech language model. It incorporates a novel alignment method, in which Kullback-Leibler divergence loss is performed to train a projector that bridges a frozen speech encoder with a frozen language model decoder. The resulting Seal model exhibits robust performance as a few-shot learner on two speech understanding tasks. Additionally, consistency experiments are conducted to validate its robustness on different pre-trained language models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
