VoxRole：一个用于评估基于语音的角色扮演智能体的综合基准
VoxRole: A Comprehensive Benchmark for Evaluating Speech-Based Role-Playing Agents
AI总结：
针对当前角色扮演对话智能体研究忽略语音副语言特征、语音角色扮演领域缺乏标准化评估基准的问题，提出首个专门的综合基准VoxRole及两阶段构建流水线，并基于该基准评估现有口语对话模型的人设一致性表现。
摘要：
大语言模型（LLMs）近期的显著进展极大推动了角色扮演对话智能体（RPCAs）的发展，这类系统旨在通过稳定的人设呈现打造沉浸式用户体验。然而当前RPCA研究存在双重局限：其一，现有研究主要聚焦文本模态，完全忽略了语音中语调、韵律、节奏等关键副语言特征，而这些特征对传递角色情绪、塑造鲜活身份至关重要；其二，基于语音的角色扮演领域长期缺乏标准化评估基准，现有多数口语对话数据集仅针对基础能力评估，角色设定单薄或定义模糊，无法有效量化模型在长期人设一致性等核心能力上的表现。为填补这一关键空白，我们提出VoxRole——首个专门用于评估基于语音的RPCA的综合基准。该基准包含13335段多轮对话，总计65.6小时语音，涵盖261部电影中的1228个独特角色。为构建该资源，我们提出一种新颖的两阶段自动化流水线：先将电影音频与剧本对齐，再利用LLM为每个角色系统地构建多维度人设。借助VoxRole，我们对当前口语对话模型开展多维度评估，揭示了其在维持人设一致性方面各自优势与局限的关键洞见。
英文摘要：
Recent significant advancements in Large Language Models (LLMs) have greatly propelled the development of Role-Playing Conversational Agents (RPCAs). These systems aim to create immersive user experiences through consistent persona adoption. However, current RPCA research faces dual limitations. First, existing work predominantly focuses on the textual modality, entirely overlooking critical paralinguistic features including intonation, prosody, and rhythm in speech, which are essential for conveying character emotions and shaping vivid identities. Second, the speech-based role-playing domain suffers from a long-standing lack of standardized evaluation benchmarks. Most current spoken dialogue datasets target only fundamental capability assessments, featuring thinly sketched or ill-defined character profiles. Consequently, they fail to effectively quantify model performance on core competencies like long-term persona consistency. To address this critical gap, we introduce VoxRole, the first comprehensive benchmark specifically designed for the evaluation of speech-based RPCAs. The benchmark comprises 13335 multi-turn dialogues, totaling 65.6 hours of speech from 1228 unique characters across 261 movies. To construct this resource, we propose a novel two-stage automated pipeline that first aligns movie audio with scripts and subsequently employs an LLM to systematically build multi-dimensional profiles for each character. Leveraging VoxRole, we conduct a multi-dimensional evaluation of contemporary spoken dialogue models, revealing crucial insights into their respective strengths and limitations in maintaining persona consistency.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
