arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2401.08276cs.CVcs.CL
收藏

AesBench:面向多模态大语言模型图像美学感知的专家评测基准

AesBench: An Expert Benchmark for Multimodal Large Language Models on Image Aesthetics Perception

Xidian University(西安电子科技大学);Nanyang Technological University(南洋理工大学);OPPO Research Institute(OPPO研究院)

Yipo Huang, Quan Yuan, Xiangfei Sheng, Zhichao Yang, Haoning Wu, Pengfei Chen, Yuzhe Yang, Leida Li, Weisi Lin

AI总结:

本文提出专家评测基准 AesBench,通过专家标注数据库 EAPD 和感知、共情、评价、解释四维标准,系统评估并揭示现有多模态大语言模型在图像美学感知上与人类仍存在显著差距。

摘要:

在多方共同努力下,多模态大语言模型(MLLMs)正经历蓬勃发展。然而,它们在图像美学感知上的表现仍不明确,而这一能力在实际应用中具有很高需求。一个明显的障碍在于缺乏专门的评测基准来评估 MLLMs 在美学感知上的有效性。这种盲目摸索可能会阻碍具备美学感知能力的更先进 MLLMs 的进一步发展。为解决这一困境,我们提出 AesBench,一个专家评测基准,旨在通过双方面的精心设计全面评估 MLLMs 的美学感知能力。(1)我们构建了一个专家标注的美学感知数据库(EAPD),该数据库具有多样化的图像内容,并提供由专业美学专家给出的高质量标注。(2)我们提出一套综合性标准,从四个视角衡量 MLLMs 的美学感知能力,包括感知(AesP)、共情(AesE)、评价(AesA)和解释(AesI)。大量实验结果表明,当前的 MLLMs 仅具备初步的美学感知能力,MLLMs 与人类之间仍存在显著差距。我们希望这项工作能够启发社区对 MLLMs 的美学潜力展开更深入的探索。源数据将在 https://github.com/yipoh/AesBench 提供。

英文摘要:

With collective endeavors, multimodal large language models (MLLMs) are undergoing a flourishing development. However, their performances on image aesthetics perception remain indeterminate, which is highly desired in real-world applications. An obvious obstacle lies in the absence of a specific benchmark to evaluate the effectiveness of MLLMs on aesthetic perception. This blind groping may impede the further development of more advanced MLLMs with aesthetic perception capacity. To address this dilemma, we propose AesBench, an expert benchmark aiming to comprehensively evaluate the aesthetic perception capacities of MLLMs through elaborate design across dual facets. (1) We construct an Expert-labeled Aesthetics Perception Database (EAPD), which features diversified image contents and high-quality annotations provided by professional aesthetic experts. (2) We propose a set of integrative criteria to measure the aesthetic perception abilities of MLLMs from four perspectives, including Perception (AesP), Empathy (AesE), Assessment (AesA) and Interpretation (AesI). Extensive experimental results underscore that the current MLLMs only possess rudimentary aesthetic perception ability, and there is still a significant gap between MLLMs and humans. We hope this work can inspire the community to engage in deeper explorations on the aesthetic potentials of MLLMs. Source data will be available at https://github.com/yipoh/AesBench.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑