Vi(E)va LLM！用于评估和解释基于生成式AI的可视化的概念堆栈
Vi(E)va LLM! A Conceptual Stack for Evaluating and Interpreting Generative AI-based Visualizations
AI总结：
本文提出EvaLLM评估堆栈，将LLM生成可视化的评估分解为原子组件，并构建平台支持自动与人工评分，通过两个案例验证其有效性。
摘要：
自动生成可视化是一个古老的任务，多年来，它已引起研究界和从业者社区越来越多的兴趣。最近，大型语言模型（LLM）已成为支持与可视化相关的生成任务的一个有趣选择，并展示了初步的有希望的结果。与此同时，一些陷阱，如指导LLM生成期望结果的多种方式、导致生成的不同视角（基于代码、基于图像、基于语法），以及即使在可视化生成任务中也存在的幻觉，使得它们的使用不如预期那样经济实惠。遵循类似的对LLM进行基准测试的倡议，本文解决了通过LLM对生成的可视化进行建模评估的问题。我们提出了一个理论评估堆栈EvaLLM，它将评估工作分解为其原子组件，表征其性质，并提供如何实现和解释它们的概述。我们还设计并实现了一个评估平台，为可视化生成任务提供基准测试资源。该平台支持由多个评估者进行的自动和手动评分，以支持基于EvaLLM堆栈的细粒度、语义评估。在GPT3.5-turbo与代码解释器和Llama2-70-b模型上的两个案例研究显示了EvaLLM的好处，并展示了当前最先进的LLM生成可视化的有趣结果。
英文摘要：
The automatic generation of visualizations is an old task that, through the years, has shown more and more interest from the research and practitioner communities. Recently, large language models (LLM) have become an interesting option for supporting generative tasks related to visualization, demonstrating initial promising results. At the same time, several pitfalls, like the multiple ways of instructing an LLM to generate the desired result, the different perspectives leading the generation (code-based, image-based, grammar-based), and the presence of hallucinations even for the visualization generation task, make their usage less affordable than expected. Following similar initiatives for benchmarking LLMs, this paper copes with the problem of modeling the evaluation of a generated visualization through an LLM. We propose a theoretical evaluation stack, EvaLLM, that decomposes the evaluation effort in its atomic components, characterizes their nature, and provides an overview of how to implement and interpret them. We also designed and implemented an evaluation platform that provides a benchmarking resource for the visualization generation task. The platform supports automatic and manual scoring conducted by multiple assessors to support a fine-grained and semantic evaluation based on the EvaLLM stack. Two case studies on GPT3.5-turbo with Code Interpreter and Llama2-70-b models show the benefits of EvaLLM and illustrate interesting results on the current state-of-the-art LLM-generated visualizations.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
