arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2407.10972cs.CVcs.AIcs.LG
收藏

VGBench:评估大型语言模型在矢量图形理解与生成方面的能力

VGBench: Evaluating Large Language Models on Vector Graphics Understanding and Generation

University of Wisconsin-Madison(威斯康星大学麦迪逊分校)

Bocheng Zou, Mu Cai, Jianrui Zhang, Yong Jae Lee

AI总结:

提出VGBench基准,通过4279个理解和5845个生成样本,全面评估LLMs在矢量图形理解与生成方面的能力,发现其在多方面表现强劲但在SVG等低级格式上表现欠佳。

摘要:

在视觉模型领域,主要的表示方式是使用像素对视觉世界进行光栅化。然而,这并不总是最好或唯一的方法,特别是对于使用多边形等几何基元描绘世界的设计师和艺术家而言。另一方面,矢量图形(VG)提供了视觉内容的文本表示,对于卡通、草图和科学图表等内容,这种方式可能更加简洁和强大。最近的研究表明,有能力的大型语言模型(LLMs)在处理矢量图形方面取得了令人鼓舞的结果。然而,这些工作仅关注定性结果、理解或特定类型的矢量图形。我们提出了VGBench,这是一个针对LLMs处理矢量图形能力的综合基准,涵盖多个方面,包括(a)视觉理解和生成,(b)评估各种矢量图形格式,(c)多样化的问题类型,(d)广泛的提示技术,(e)在多个LLMs下,以及(f)与VLMs在光栅化表示上的比较。在我们收集的4279个理解样本和5845个生成样本上进行评估,我们发现LLMs在两个方面都展现出强大的能力,但在低级格式(SVG)上的表现则不太理想。数据和评估流程将在 https://vgbench.github.io 开源。

英文摘要:

In the realm of vision models, the primary mode of representation is using pixels to rasterize the visual world. Yet this is not always the best or unique way to represent visual content, especially for designers and artists who depict the world using geometry primitives such as polygons. Vector graphics (VG), on the other hand, offer a textual representation of visual content, which can be more concise and powerful for content like cartoons, sketches and scientific figures. Recent studies have shown promising results on processing vector graphics with capable Large Language Models (LLMs). However, such works focus solely on qualitative results, understanding, or a specific type of vector graphics. We propose VGBench, a comprehensive benchmark for LLMs on handling vector graphics through diverse aspects, including (a) both visual understanding and generation, (b) evaluation of various vector graphics formats, (c) diverse question types, (d) wide range of prompting techniques, (e) under multiple LLMs and (f) comparison with VLMs on rasterized representations. Evaluating on our collected 4279 understanding and 5845 generation samples, we find that LLMs show strong capability on both aspects while exhibiting less desirable performance on low-level formats (SVG). Both data and evaluation pipeline will be open-sourced at https://vgbench.github.io.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑