利用混合专家和动态下采样增强基于多语言大模型的语音识别
Enhancing Multilingual LLM-based ASR with Mixture of Experts and Dynamic Downsampling
AI总结：
提出基于投影器的LLM-ASR框架，通过混合专家架构提升跨语言适应性，并利用连续整合-触发机制实现动态下采样和模态对齐，实验表明该方法显著超越强基线模型。
摘要：
大语言模型的快速发展为自动语音识别开辟了新前沿，使其有效集成成为一个关键且具有挑战性的研究方向。为此，本文提出了一种基于投影器的LLM-ASR框架，针对多语言泛化和模态对齐的关键挑战。我们的方法结合了混合专家架构以改善跨语言适应性，以及连续整合-触发机制用于动态下采样和模态对齐。实验结果表明，这些组件的组合带来了显著的性能提升，超越了强基线模型。所提出的方法朝着构建更准确、更鲁棒、更泛化的基于LLM的ASR系统迈出了一步。
英文摘要：
The rapid progress of large language models (LLMs) has opened up a new frontier for automatic speech recognition (ASR), making their effective integration a critical and challenging research direction. To this end, this work proposes a projector-based LLM-ASR framework targeting the key challenges of multilingual generalization and modality alignment. Our approach incorporates a Mixture of Experts (MoE) architecture to improve cross-lingual adaptability, and a Continuous Integrate-and-Fire (CIF) mechanism for dynamic downsampling and modality alignment. Experimental results show that the combination of these components yields substantial performance improvements, surpassing strong baseline models. The proposed method represents a step toward building more accurate, robust, and generalizable LLM-based ASR systems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
