arXiv 2512.013402025-12-01cs.CV

EvalTalker: 学习评估基于真实人像的多主体说话人类

EvalTalker: Learning to Evaluate Real-Portrait-Driven Multi-Subject Talking Humans

Shanghai Jiao Tong University（上海交通大学）；Meituan（美团）；Shanghai Innovation Institute（上海创新研究院）

Yingjie Zhou, Xilei Zhu, Siyu Ren, Ziyi Zhao, Ziwen Wang, Farong Wen, Yu Zhou, Jiezhang Cao, Xiongkuo Min, Fengjiao Chen, Xiaoyu Li, Xuezhi Cao, Guangtao Zhai, Xiaohong Liu

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

EvalTalker通过构建大规模多Talker生成的说话人类质量评估数据集，提出了一种能够感知整体质量、人类特征和身份一致性的新型质量评估框架，以提升多主体说话人类生成的质量和用户体验。

摘要：

语音驱动的说话人类（TH）生成，通常称为“Talker”，目前在多主体驱动能力方面存在局限性。将这一范式扩展到“多Talker”，能够同时动画化多个主体，引入更丰富的交互性和更强的沉浸感在视听交流中。然而，当前的多Talker仍然表现出明显的质量退化，这是由于技术限制导致的，从而导致用户体验不佳。为了解决这一挑战，我们构建了THQA-MT，第一个大规模的多Talker生成的说话人类质量评估数据集，包含来自15个代表性多Talker的5,492个多Talker生成的TH（MTHs），使用在线收集的400个真实人像。通过主观实验，我们分析了不同多Talker之间的感知差异，并确定了12种常见的失真类型。此外，我们引入了EvalTalker，一种新的TH质量评估框架。该框架能够感知整体质量、人类特征和身份一致性，同时整合Qwen-Sync来感知多模态同步性。实验结果表明，EvalTalker在与主观评分的相关性上表现优异，为未来高质量多Talker生成和评估研究提供了稳健的基础。

英文摘要：

Speech-driven Talking Human (TH) generation, commonly known as "Talker," currently faces limitations in multi-subject driving capabilities. Extending this paradigm to "Multi-Talker," capable of animating multiple subjects simultaneously, introduces richer interactivity and stronger immersion in audiovisual communication. However, current Multi-Talkers still exhibit noticeable quality degradation caused by technical limitations, resulting in suboptimal user experiences. To address this challenge, we construct THQA-MT, the first large-scale Multi-Talker-generated Talking Human Quality Assessment dataset, consisting of 5,492 Multi-Talker-generated THs (MTHs) from 15 representative Multi-Talkers using 400 real portraits collected online. Through subjective experiments, we analyze perceptual discrepancies among different Multi-Talkers and identify 12 common types of distortion. Furthermore, we introduce EvalTalker, a novel TH quality assessment framework. This framework possesses the ability to perceive global quality, human characteristics, and identity consistency, while integrating Qwen-Sync to perceive multimodal synchrony. Experimental results demonstrate that EvalTalker achieves superior correlation with subjective scores, providing a robust foundation for future research on high-quality Multi-Talker generation and evaluation.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。