MENASpeechBank: 一个具有基于人设的多轮对话的参考语音库用于音频大语言模型
MENASpeechBank: A Reference Voice Bank with Persona-Conditioned Multi-Turn Conversations for AudioLLMs
AI总结：
MENASpeechBank通过合成数据管道生成多轮对话数据，支持音频大语言模型在多样化语音和方言场景中的训练与应用。
摘要：
音频大语言模型（AudioLLMs）能够对语音和通用音频进行指令遵循，但进展日益受到缺乏多样、对话式、指令对齐的语音-文本数据的限制。这个瓶颈在基于人设的交互和方言覆盖方面尤为突出，其中收集和发布真实多说话者录音成本高且速度慢。我们引入了MENASpeechBank，一个包含约18,000个高质量陈述的参考语音库，涵盖来自多个中东国家的124名说话者，涵盖英语、现代标准阿拉伯语（MSA）和地方阿拉伯语变体。基于这一资源，我们开发了一条可控的合成数据管道，包括：（i）构建包含世界价值观调查启发属性的详细人设档案；（ii）定义约5,000个对话场景的分类学；（iii）通过语义相似性将人设与场景匹配；（iv）生成约417,000个角色扮演对话，其中用户以人设身份说话，助手以帮助代理身份回应；（v）通过参考说话人音频条件生成用户部分，以保持说话人身份和多样性。我们评估了合成和真人录制的对话，并提供了详细的分析。我们将发布MENASpeechBank和生成的对话，供社区使用。
英文摘要：
Audio large language models (AudioLLMs) enable instruction following over speech and general audio, but progress is limited by the scarcity of diverse, conversational, and instruction-aligned speech--text data. This gap is particularly pronounced for persona-grounded and dialectal interactions, where collecting real multi-speaker recordings remains costly and slow. We introduce MENASpeechBank, a reference speech bank comprising ~18K high-quality utterances from 124 speakers spanning multiple MENA countries, covering English, Modern Standard Arabic (MSA), and regional Arabic varieties. We develop a controllable data pipeline that (i) constructs persona profiles enriched with World Values Survey (WVS) inspired attributes, (ii) defines a taxonomy driven ~5Kconversational scenarios, (iii) matches personas to scenarios via semantic similarity, (iv) generates ~417K role-play conversations with an LLM where the user speaks as the persona and the assistant behaves as a helpful agent, and (v) produces speaker-conditioned user-turn audio (synthetic) from reference recordings to preserve speaker diversity. We evaluate synthetic and human recorded conversations and provide an analysis. We will make the MENASpeechBank available for the community.(\href{https://huggingface.co/datasets/QCRI/MenaSpeechBank)
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
