LMME3DHF：基于大语言多模态模型的3D人脸生成基准与评估
LMME3DHF: Benchmarking and Evaluating Multimodal 3D Human Face Generation with LMMs
AI总结：
针对AI生成3D人脸质量评估难题，提出Gen3DHF基准（含2000视频、4000MOS）及LMME3DHF多模态评估指标，实现质量/真实性预测、失真问答与显著性预测，性能超越现有方法。
摘要：
生成式人工智能的快速发展使得3D人脸（HFs）的创建成为可能，其应用涵盖媒体制作、虚拟现实、安全、医疗保健和游戏开发等领域。然而，由于人类感知的主观性以及对面部特征天生的感知敏感性，评估这些AI生成的3D人脸的质量和真实感仍然是一个重大挑战。为此，我们对AI生成的3D人脸的质量评估进行了全面研究。我们首先引入了Gen3DHF，这是一个大规模基准，包含2,000个AI生成的3D人脸视频，以及从两个维度（即质量和真实性）收集的4,000个平均意见分数（MOS）、2,000个失真感知显著性图和失真描述。基于Gen3DHF，我们提出了LMME3DHF，一种基于大型多模态模型（LMM）的3DHF评估指标，能够进行质量和真实性分数预测、失真感知视觉问答以及失真感知显著性预测。实验结果表明，LMME3DHF达到了最先进的性能，在准确预测AI生成的3D人脸质量分数以及有效识别失真感知显著区域和失真类型方面均超越了现有方法，同时与人类感知判断保持高度一致。Gen3DHF数据库和LMME3DHF将在发表后发布。
英文摘要：
The rapid advancement in generative artificial intelligence have enabled the creation of 3D human faces (HFs) for applications including media production, virtual reality, security, healthcare, and game development, etc. However, assessing the quality and realism of these AI-generated 3D human faces remains a significant challenge due to the subjective nature of human perception and innate perceptual sensitivity to facial features. To this end, we conduct a comprehensive study on the quality assessment of AI-generated 3D human faces. We first introduce Gen3DHF, a large-scale benchmark comprising 2,000 videos of AI-Generated 3D Human Faces along with 4,000 Mean Opinion Scores (MOS) collected across two dimensions, i.e., quality and authenticity, 2,000 distortion-aware saliency maps and distortion descriptions. Based on Gen3DHF, we propose LMME3DHF, a Large Multimodal Model (LMM)-based metric for Evaluating 3DHF capable of quality and authenticity score prediction, distortion-aware visual question answering, and distortion-aware saliency prediction. Experimental results show that LMME3DHF achieves state-of-the-art performance, surpassing existing methods in both accurately predicting quality scores for AI-generated 3D human faces and effectively identifying distortion-aware salient regions and distortion types, while maintaining strong alignment with human perceptual judgments. Both the Gen3DHF database and the LMME3DHF will be released upon the publication.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
