arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.11482cs.SDcs.CLeess.AS
收藏

AnimeScore: 基于偏好的数据集与框架用于评估动漫风格语音

AnimeScore: A Preference-Based Dataset and Framework for Evaluating Anime-Like Speech Style

Spellbrush, USA(美国Spellbrush)

Joonyong Park, Jerry Li

AI总结:

针对动漫风格语音缺乏客观评估指标的问题,提出基于偏好排序的框架AnimeScore,通过187名评估者的15000对判断数据,利用声学分析和SSL排序模型实现高达90.8% AUC的自动评估。

摘要:

目前评估“动漫风格”语音依赖于昂贵的主观判断,尚无标准化的客观指标。一个关键挑战在于,与自然度不同,动漫相似度缺乏共享的绝对尺度,使得传统的平均意见得分(MOS)协议不可靠。为填补这一空白,我们提出AnimeScore,一个基于偏好的框架,通过成对排序自动评估动漫相似度。我们收集了来自187名评估者的15000对成对判断,并附有自由形式的描述;声学分析表明,感知的动漫相似度由受控的共振峰塑造、韵律连续性和刻意发音驱动,而非简单的启发式规则如高音调。我们证明,手工设计的声学特征达到69.3%的AUC上限,而基于SSL的排序模型达到90.8%的AUC,提供了一个实用的度量标准,也可作为生成式语音模型基于偏好优化的奖励信号。

英文摘要:

Evaluating 'anime-like' voices currently relies on costly subjective judgments, yet no standardized objective metric exists. A key challenge is that anime-likeness, unlike naturalness, lacks a shared absolute scale, making conventional Mean Opinion Score (MOS) protocols unreliable. To address this gap, we propose AnimeScore, a preference-based framework for automatic anime-likeness evaluation via pairwise ranking. We collect 15,000 pairwise judgments from 187 evaluators with free-form descriptions, and acoustic analysis reveals that perceived anime-likeness is driven by controlled resonance shaping, prosodic continuity, and deliberate articulation rather than simple heuristics such as high pitch. We show that handcrafted acoustic features reach a 69.3% AUC ceiling, while SSL-based ranking models achieve up to 90.8% AUC, providing a practical metric that can also serve as a reward signal for preference-based optimization of generative speech models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑