AudioRole: 一个用于大型语言模型角色扮演的音频数据集
AudioRole: An Audio Dataset for Character Role-Playing in Large Language Models
AI总结：
本文提出AudioRole数据集，通过13部电视剧1000+小时的100万+角色对话，提供同步音频文本对及角色身份标注，结合ARP-Eval评估框架，验证了GLM-4-Voice在角色扮演中的优越性能。
摘要：
高质量多模态数据集的创建对于推动大型语言模型（LLMs）中的角色扮演能力至关重要。尽管现有研究主要集中在基于文本的人设模拟，但Audio Role-Playing（ARP）由于需要同步对齐语义内容和语音特征而面临独特挑战。为此，我们提出了AudioRole，一个从13部电视剧中精心编纂的数据集，涵盖1000+小时和100万+角色相关的对话，提供同步音频-文本对，标注说话者身份和上下文元数据。此外，为了展示数据集的有效性，我们引入了ARP-Eval，一个双方面评估框架，评估响应质量和角色忠实度。实证验证显示，GLM-4-Voice在AudioRole上训练的模型（称为ARP-Model）在平均语音个性化得分上达到0.31，显著优于原始GLM-4-voice和更强大的MiniCPM-O-2.6模型。ARP-Model还实现了0.36的语音内容个性化得分，比未训练的原始模型高出约38%，与MiniCPM-O-2.6保持相同水平。AudioRole包含超过115个主要角色的对话、6个训练好的ARP-Model以及评估协议，共同为推进音频接地角色扮演研究提供了必要的资源。
英文摘要：
The creation of high-quality multimodal datasets remains fundamental for advancing role-playing capabilities in large language models (LLMs). While existing works predominantly focus on text-based persona simulation, Audio Role-Playing (ARP) presents unique challenges due to the need for synchronized alignment of semantic content and vocal characteristics. To address this gap, we propose AudioRole, a meticulously curated dataset from 13 TV series spanning 1K+ hours with 1M+ character-grounded dialogues, providing synchronized audio-text pairs annotated with speaker identities and contextual metadata. In addition, to demonstrate the effectiveness of the dataset, we introduced ARP-Eval, a dual-aspect evaluation framework that assesses both response quality and role fidelity. Empirical validation showing GLM-4-Voice trained on AudioRole (which we called ARP-Model) achieve an average Acoustic Personalization score of 0.31, significantly outperforming the original GLM-4-voice and the more powerful model MiniCPM-O-2.6, which specifically supports role-playing in one-shot scenarios. The ARP-Model also achieves a Content Personalization score of 0.36, surpassing the untrained original model by about 38% and maintaining the same level as MiniCPM-O-2.6. AudioRole features dialogues from over 115 main characters, 6 trained ARP-Models that role-play different characters, and evaluation protocols. Together, they provide an essential resource for advancing audio-grounded role-playing research.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
