通过文本提示评估推进零样本数字人质量评价
Advancing Zero-Shot Digital Human Quality Assessment through Text-Prompted Evaluation
AI总结：
针对数字人质量评价数据库缺乏问题，本文提出SJTU-H3D数据库和基于CLIP、NIQE与网格几何特征的零样本无参考评价方法DHQI，显著提升零样本性能。
摘要：
数字人已在多个领域得到广泛应用，因此需要开展相关的质量评价研究。然而，目前缺乏全面的数字人质量评价（DHQA）数据库。为填补这一空白，我们提出了 SJTU-H3D，一个专门针对全身数字人设计的主观质量评价数据库。该数据库包含 40 个高质量参考数字人以及 1,120 个带标签的失真对应样本，这些失真样本由七种失真类型生成。SJTU-H3D 数据库可作为 DHQA 研究的基准，用于评估和改进处理算法。此外，我们提出了一种零样本 DHQA 方法，该方法聚焦于无参考（NR）场景，以确保泛化能力，同时减轻数据库偏差。我们的方法利用从投影中提取的语义特征和失真特征，以及从数字人网格结构中导出的几何特征。具体而言，我们采用对比语言-图像预训练（CLIP）模型来度量语义亲和度，并结合自然图像质量评价器（NIQE）模型来捕获低层失真信息。此外，我们利用二面角作为几何描述符来提取网格特征。通过聚合这些度量，我们提出了数字人质量指数（DHQI），该指数在零样本性能上表现出显著提升。DHQI 还可以作为 DHQA 任务的稳健基线，推动该领域的发展。数据库和代码可在 https://github.com/zzc-1998/SJTU-H3D 获取。
英文摘要：
Digital humans have witnessed extensive applications in various domains, necessitating related quality assessment studies. However, there is a lack of comprehensive digital human quality assessment (DHQA) databases. To address this gap, we propose SJTU-H3D, a subjective quality assessment database specifically designed for full-body digital humans. It comprises 40 high-quality reference digital humans and 1,120 labeled distorted counterparts generated with seven types of distortions. The SJTU-H3D database can serve as a benchmark for DHQA research, allowing evaluation and refinement of processing algorithms. Further, we propose a zero-shot DHQA approach that focuses on no-reference (NR) scenarios to ensure generalization capabilities while mitigating database bias. Our method leverages semantic and distortion features extracted from projections, as well as geometry features derived from the mesh structure of digital humans. Specifically, we employ the Contrastive Language-Image Pre-training (CLIP) model to measure semantic affinity and incorporate the Naturalness Image Quality Evaluator (NIQE) model to capture low-level distortion information. Additionally, we utilize dihedral angles as geometry descriptors to extract mesh features. By aggregating these measures, we introduce the Digital Human Quality Index (DHQI), which demonstrates significant improvements in zero-shot performance. The DHQI can also serve as a robust baseline for DHQA tasks, facilitating advancements in the field. The database and the code are available at https://github.com/zzc-1998/SJTU-H3D.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
