arXiv 2601.124942026-07-08cs.SDcs.AIcs.CLeess.AS

通过数据调度实现低资源阿拉伯语音频LLM的多任务指令微调

Multi-Task Instruction Tuning via Data Scheduling for Low-Resource Arabic SpeechLLMs

Qatar Computing Research Institute（卡塔尔计算研究院）

Hunzalah Hassan Bhatti, Firoj Alam, Shammur Absar Chowdhury

备注：: Foundation Models, Large Language Models, Native, Speech Models, Arabic

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文研究了在资源受限环境下，通过数据调度优化多任务指令微调方法，提出AraMega-SSum用于训练和评估阿拉伯语中心的音频LLM，并比较了四种训练策略，发现两阶段TPC→ADS策略在任务平衡上表现最佳。

摘要：

音频大语言模型（LLM）能够实现统一的语音理解和生成，但将其适应到语言复杂且方言丰富的环境，如阿拉伯-英语，仍具挑战性。本文针对阿拉伯语为中心的音频LLM，进行了多任务指令微调的受控研究，涵盖生成任务如ASR和语音与文本摘要，以及判别任务如方言和情感识别。为支持端到端的阿拉伯语语音摘要，我们引入了AraMega-SSum，这是首个用于训练和评估阿拉伯语中心音频LLM的语音摘要资源。我们比较了四种训练策略：（i）均匀任务混合，（ii）任务渐进课程（TPC），（iii）基于对齐器的多样化采样（ADS）用于训练时间批量构建，以及（iv）两阶段TPC→ADS策略。我们的结果表明存在明显的效率-鲁棒性权衡。ADS加速了早期收敛并提高了副语言性能，但损害了其他任务。两阶段TPC→ADS策略在任务平衡上表现最佳，为适应低资源、方言丰富的环境提供了实用指导。我们将在社区中公开AraMega-SSum和所有实验资源。

英文摘要：

Audio large language models (LLMs) enable unified speech understanding and generation, but adapting them to linguistically complex and dialect-rich settings such as Arabic-English remains challenging. We present a controlled study of multi-task instruction tuning for an Arabic-centric audio LLM across generative tasks, including automatic speech recognition (ASR) and speech and text summarization, as well as discriminative tasks, including dialect identification (DID) and speech emotion recognition (SER), in a resource-constrained setting. To support end-to-end Arabic speech summarization, we introduce AraMega-SSum, the first Arabic speech summarization dataset designed for training and benchmarking Arabic-centric audio LLMs. We compare four training strategies: (i) Uniform Mixing (UM), (ii) Task-Progressive Curriculum (TPC), (iii) Aligner-Based Diverse Sampling (ADS) for training-time batch construction, and (iv) a two-stage TPC->ADS strategy. Our results reveal a clear efficiency-robustness trade-off. TPC achieves the strongest performance on generative tasks, including ASR and summarization. ADS improves paralinguistic tasks but reduces generative stability when used alone. The two-stage TPC->ADS strategy provides the best overall balance, achieving the strongest DID and SER performance while outperforming large proprietary models such as Gemini-2.5-Pro on discriminative tasks. We will publicly release AraMega-SSum together with all experimental resources to support future research in Arabic speech understanding.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。