Lexara: 一种以用户为中心的评估工具包，用于评估用于对话式可视化分析的大型语言模型
Lexara: A User-Centered Toolkit for Evaluating Large Language Models for Conversational Visual Analytics
AI总结：
Lexara是一种以用户为中心的评估工具包，用于评估对话式可视化分析中的大型语言模型，通过多格式输出的可解释指标和交互式工具帮助用户进行模型和提示选择。
摘要：
大型语言模型（LLMs）通过自然语言使对话式可视化分析（CVA）发生变革，通过使数据分析成为可能。然而，评估LLMs用于CVA仍是一个挑战：需要编程专业知识，忽视现实世界的复杂性，缺乏可解释的指标用于多格式（可视化和文本）输出。通过与22名CVA开发者和16名终端用户的访谈，我们确定了用例、评估标准和工作流程。我们提出了Lexara，一种以用户为中心的CVA评估工具包，将这些见解转化为：（i）覆盖现实场景的测试用例；（ii）涵盖可视化质量（数据保真度、语义对齐、功能正确性、设计清晰度）和语言质量（事实基础、分析推理、对话连贯性）的可解释指标，使用基于规则和LLM-as-a-Judge的方法；以及（iii）一个交互式工具包，使实验设置和多格式和多级结果探索成为可能，而无需编程专业知识。我们与六名CVA开发者进行了为期两周的日记研究，这些开发者来自我们最初的22人小组。他们的反馈证明了Lexara在指导适当模型和提示选择方面的有效性。
英文摘要：
Large Language Models (LLMs) are transforming Conversational Visual Analytics (CVA) by enabling data analysis through natural language. However, evaluating LLMs for CVA remains a challenge: requiring programming expertise, overlooking real-world complexity, and lacking interpretable metrics for multi-format (visualizations and text) outputs. Through interviews with 22 CVA developers and 16 end-users, we identified use cases, evaluation criteria and workflows. We present Lexara, a user-centered evaluation toolkit for CVA that operationalizes these insights into: (i) test cases spanning real-world scenarios; (ii) interpretable metrics covering visualization quality (data fidelity, semantic alignment, functional correctness, design clarity) and language quality (factual grounding, analytical reasoning, conversational coherence) using rule-based and LLM-as-a-Judge methods; and (iii) an interactive toolkit enabling experimental setup and multi-format and multi-level exploration of results without programming expertise. We conducted a two-week diary study with six CVA developers, drawn from our initial cohort of 22. Their feedback demonstrated Lexara's effectiveness for guiding appropriate model and prompt selection.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
