探索有限语音数据下大型音频语言模型的微调以提升语音语言理解
Exploring Fine-Tuning of Large Audio Language Models for Spoken Language Understanding under Limited Speech Data
AI总结：
本研究探讨了在有限语音数据条件下，通过不同微调方案提升语音语言理解性能，发现文本-only微调已具竞争力，课程学习在数据稀缺时效果显著。
摘要：
大型音频语言模型（LALMs）已涌现为语音相关任务的强大工具，但在有限语音数据下的微调仍处于探索阶段。为弥合这一差距，我们系统地检验了不同的微调方案，包括文本-only、直接混合和课程学习，以影响语音语言理解（SLU），重点研究在文本标签对丰富的场景下，而配对语音标签数据有限的情况。结果表明，LALMs在文本-only微调下已能取得具有竞争力的性能，突显其强大的泛化能力。即使添加少量语音数据（2-5%）也能带来显著的进一步提升，其中课程学习在数据稀缺时特别有效。在跨语言SLU中，结合源语言语音数据与目标语言文本和少量目标语言语音数据可实现有效的适应。总体而言，本研究为在现实数据约束下LALM的微调提供了实用见解。
英文摘要：
Large Audio Language Models (LALMs) have emerged as powerful tools for speech-related tasks but remain underexplored for fine-tuning, especially with limited speech data. To bridge this gap, we systematically examine how different fine-tuning schemes including text-only, direct mixing, and curriculum learning affect spoken language understanding (SLU), focusing on scenarios where text-label pairs are abundant while paired speech-label data are limited. Results show that LALMs already achieve competitive performance with text-only fine-tuning, highlighting their strong generalization ability. Adding even small amounts of speech data (2-5%) yields substantial further gains, with curriculum learning particularly effective under scarce data. In cross-lingual SLU, combining source-language speech data with target-language text and minimal target-language speech data enables effective adaptation. Overall, this study provides practical insights into the LALM fine-tuning under realistic data constraints.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
