S-VoCAL:用于推断文学作品中说话声音特征属性的数据集和评估框架
S-VoCAL: A Dataset and Evaluation Framework for Inferring Speaking Voice Character Attributes in Literature
AI总结:
S-VoCAL通过引入首个专门评估声音相关虚构角色属性推断的数据集和框架,展示了RAG流程在推断年龄和性别等属性上的有效性,但在出身和身体健康等属性上存在挑战。
摘要:
随着文本到语音(TTS)系统的发展,合成有声书叙述受到了越来越多的关注,其自然度达到了前所未有的水平。然而,合成叙述系统在模仿虚构角色和传达复杂情感或语调方面仍存在较大的差距。一个有前景的方向是为每部小说中的虚构角色分配合理的声音。这一步通常需要在长篇小说的上下文中进行复杂的属性推断,例如角色的年龄、性别、出身或身体健康状况,这反过来又需要专门的基准数据集来评估提取系统的性能。我们提出了S-VoCAL(文学作品中的说话声音特征属性),这是首个专门用于评估声音相关虚构角色属性推断的数据库和评估框架。S-VoCAL包含8个基于社会语音学研究的属性,以及从项目古腾堡计划中获得的952个角色-书籍配对。其评估框架针对每个属性的特殊性,并包含基于最近大型语言模型嵌入的新型相似性度量。我们通过应用一个简单的检索增强生成(RAG)流程到推断角色属性的任务中,证明了S-VoCAL的适用性。我们的结果表明,RAG流程能够可靠地推断出年龄或性别等属性,但在推断出身或身体健康状况等属性时则表现较差。数据集和评估代码可在https://github.com/AbigailBerthe/S-VoCAL上获取。
英文摘要:
With recent advances in Text-to-Speech (TTS) systems, synthetic audiobook narration has seen increased interest, reaching unprecedented levels of naturalness. However, larger gaps remain in synthetic narration systems' ability to impersonate fictional characters, and convey complex emotions or prosody. A promising direction to enhance character identification is the assignment of plausible voices to each fictional characters in a book. This step typically requires complex inference of attributes in book-length contexts, such as a character's age, gender, origin or physical health, which in turns requires dedicated benchmark datasets to evaluate extraction systems' performances. We present S-VoCAL (Speaking Voice Character Attributes in Literature), the first dataset and evaluation framework dedicated to evaluate the inference of voice-related fictional character attributes. S-VoCAL entails 8 attributes grounded in sociophonetic studies, and 952 character-book pairs derived from Project Gutenberg. Its evaluation framework addresses the particularities of each attribute, and includes a novel similarity metric based on recent Large Language Models embeddings. We demonstrate the applicability of S-VoCAL by applying a simple Retrieval-Augmented Generation (RAG) pipeline to the task of inferring character attributes. Our results suggest that the RAG pipeline reliably infers attributes such as Age or Gender, but struggles on others such as Origin or Physical Health. The dataset and evaluation code are available at https://github.com/AbigailBerthe/S-VoCAL .
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。