绘制未来：利用图表问答对LLM驱动的数据可视化进行可扩展评估
Charting the Future: Using Chart Question-Answering for Scalable Evaluation of LLM-Driven Data Visualizations
AI总结：
本文提出利用视觉问答模型自动评估LLM生成的数据可视化质量，实验表明少样本提示能显著提升图表生成准确性，但LLM生成图表仍不及人工生成图表的精确度。
摘要：
我们提出了一个新颖的框架，利用视觉问答（VQA）模型来自动化评估由大语言模型（LLM）生成的数据可视化。传统的评估方法通常依赖人工判断，这既昂贵又不可扩展，或者仅关注数据准确性，而忽视了视觉传达的有效性。通过采用VQA模型，我们评估了数据表示质量以及图表的整体沟通清晰度。实验使用两个领先的VQA基准数据集ChartQA和PlotQA进行，可视化图表由OpenAI的GPT-3.5 Turbo和Meta的Llama 3.1 70B-Instruct模型生成。我们的结果表明，基于VQA性能指标，LLM生成的图表在准确性上不及原始的非LLM生成图表。此外，虽然我们的结果表明少样本提示（few-shot prompting）显著提升了图表生成的准确性，但在LLM能够完全达到人工生成图表的精确度之前，仍有相当大的进步空间。这凸显了我们工作的重要性，该工作通过实现无需人工标注的快速迭代来加快研究进程，从而加速该领域的进展。
英文摘要：
We propose a novel framework that leverages Visual Question Answering (VQA) models to automate the evaluation of LLM-generated data visualizations. Traditional evaluation methods often rely on human judgment, which is costly and unscalable, or focus solely on data accuracy, neglecting the effectiveness of visual communication. By employing VQA models, we assess data representation quality and the general communicative clarity of charts. Experiments were conducted using two leading VQA benchmark datasets, ChartQA and PlotQA, with visualizations generated by OpenAI's GPT-3.5 Turbo and Meta's Llama 3.1 70B-Instruct models. Our results indicate that LLM-generated charts do not match the accuracy of the original non-LLM-generated charts based on VQA performance measures. Moreover, while our results demonstrate that few-shot prompting significantly boosts the accuracy of chart generation, considerable progress remains to be made before LLMs can fully match the precision of human-generated graphs. This underscores the importance of our work, which expedites the research process by enabling rapid iteration without the need for human annotation, thus accelerating advancements in this field.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
