DracoGPT:从大型语言模型中提取可视化设计偏好
DracoGPT: Extracting Visualization Design Preferences from Large Language Models
AI总结:
本文提出DracoGPT方法,通过DracoGPT-Rank和DracoGPT-Recommend两条流程提取并建模LLMs中的可视化设计偏好,发现其与人类实验准则存在显著偏差,为扩展Draco知识库和替代LLMs提供基础。
摘要:
在大量语料上训练的大型语言模型(LLMs)有潜力编码可视化设计知识和最佳实践。然而,如果它们未能做到这一点,就可能提供不可靠的可视化建议。那么,LLMs究竟学到了哪些可视化设计偏好?我们提出了DracoGPT,一种从LLMs中提取、建模和评估可视化设计偏好的方法。为了评估不同任务,我们开发了两条流程——DracoGPT-Rank和DracoGPT-Recommend——用于对LLMs进行提示建模,使其对可视化编码规范进行排序或推荐。我们使用Draco作为共享知识库来表示LLM的设计偏好,并将其与实证研究得出的最佳实践进行比较。我们证明DracoGPT能够准确建模LLMs表达的偏好,从而能够依据Draco设计约束进行分析。在一组后端LLMs中,我们发现DracoGPT-Rank和DracoGPT-Recommend彼此之间具有中等程度的一致性,但两者都明显偏离了来自人类受试者实验的指导原则。未来的工作可以在我们的方法基础上扩展Draco的知识库,以建模更丰富的偏好集合,并为LLMs提供稳健且经济高效的替代方案。
英文摘要:
Trained on vast corpora, Large Language Models (LLMs) have the potential to encode visualization design knowledge and best practices. However, if they fail to do so, they might provide unreliable visualization recommendations. What visualization design preferences, then, have LLMs learned? We contribute DracoGPT, a method for extracting, modeling, and assessing visualization design preferences from LLMs. To assess varied tasks, we develop two pipelines--DracoGPT-Rank and DracoGPT-Recommend--to model LLMs prompted to either rank or recommend visual encoding specifications. We use Draco as a shared knowledge base in which to represent LLM design preferences and compare them to best practices from empirical research. We demonstrate that DracoGPT can accurately model the preferences expressed by LLMs, enabling analysis in terms of Draco design constraints. Across a suite of backing LLMs, we find that DracoGPT-Rank and DracoGPT-Recommend moderately agree with each other, but both substantially diverge from guidelines drawn from human subjects experiments. Future work can build on our approach to expand Draco's knowledge base to model a richer set of preferences and to provide a robust and cost-effective stand-in for LLMs.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。