评估大语言模型在数据可视化自然语言话语中的语义剖析能力
Evaluating the Semantic Profiling Abilities of LLMs for Natural Language Utterances in Data Visualization
AI总结：
本研究评估了四款公开大语言模型在数据可视化自然语言话语中的语义剖析能力，发现其对话语不确定性敏感但能提取数据上下文，却难以推断可视化任务，并指明了未来研究方向。
摘要：
针对数据集上的人类话语自动生成数据可视化，需要深度理解数据话语的语义，包括对数据属性、可视化任务及必要数据准备步骤的隐式和显式引用。用于数据可视化的自然语言界面（NLIs）已探索了多种推断此类信息的方法，但由于人类语言固有的不确定性，挑战依然存在。大语言模型（LLMs）的最新进展为解决这些挑战提供了途径，但其提取相关语义信息的能力仍未得到探索。本研究评估了四个公开可用的LLM（GPT-4、Gemini-Pro、Llama3和Mixtral），研究它们即使在存在不确定性的情况下理解话语、识别相关数据上下文和可视化任务的能力。研究结果表明，LLM对话语中的不确定性很敏感。尽管存在这种敏感性，它们仍能提取相关的数据上下文。然而，LLM在推断可视化任务方面存在困难。基于这些结果，我们指出了利用LLM进行可视化生成的未来研究方向。
英文摘要：
Automatically generating data visualizations in response to human utterances on datasets necessitates a deep semantic understanding of the data utterance, including implicit and explicit references to data attributes, visualization tasks, and necessary data preparation steps. Natural Language Interfaces (NLIs) for data visualization have explored ways to infer such information, yet challenges persist due to inherent uncertainty in human speech. Recent advances in Large Language Models (LLMs) provide an avenue to address these challenges, but their ability to extract the relevant semantic information remains unexplored. In this study, we evaluate four publicly available LLMs (GPT-4, Gemini-Pro, Llama3, and Mixtral), investigating their ability to comprehend utterances even in the presence of uncertainty and identify the relevant data context and visual tasks. Our findings reveal that LLMs are sensitive to uncertainties in utterances. Despite this sensitivity, they are able to extract the relevant data context. However, LLMs struggle with inferring visualization tasks. Based on these results, we highlight future research directions on using LLMs for visualization generation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
