arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2403.04758cs.HCcs.AIcs.CYcs.LG
收藏

KnowledgeVIS:通过比较填空提示解读语言模型

KnowledgeVIS: Interpreting Language Models by Comparing Fill-in-the-Blank Prompts

Georgia Institute of Technology(佐治亚理工学院)

Adam Coscia, Alex Endert

AI总结:

为帮助理解大语言模型的工作原理,本研究提出基于填空提示的人在回路视觉分析系统KnowledgeVIS,支持提示测试、语义聚类分析与交互式可视化,并通过专家反馈和三类用例验证了其有效性。

摘要:

近年来大语言模型的日益普及使其被越来越多地用于文本摘要、预测和生成,因此帮助研究人员和工程师理解其工作方式和原理至关重要。我们提出KnowledgeVIS,这是一套人在回路的视觉分析系统,采用填空句作为提示来解读语言模型。通过对比不同句子间的预测结果,KnowledgeVIS可揭示模型习得的关联,直观地将语言模型在训练中学到的知识与下游自然语言任务关联起来,还能帮助用户创建并测试多种提示变体,借助一种新型语义聚类技术分析预测词汇,并通过交互式可视化挖掘洞见。这些可视化功能整体可帮助用户判断单个预测的可能性与独特性,对比不同提示的预测集合,以及总结所有提示下预测结果的模式与关联。我们结合6位NLP专家的反馈以及三个不同用例验证了KnowledgeVIS的功能:(1)在两个领域适配模型中探究生物医学知识;(2)评估有害的身份刻板印象;(3)在三个通用模型中发现事实与关联。

英文摘要:

Recent growth in the popularity of large language models has led to their increased usage for summarizing, predicting, and generating text, making it vital to help researchers and engineers understand how and why they work. We present KnowledgeVis, a human-in-the-loop visual analytics system for interpreting language models using fill-in-the-blank sentences as prompts. By comparing predictions between sentences, KnowledgeVis reveals learned associations that intuitively connect what language models learn during training to natural language tasks downstream, helping users create and test multiple prompt variations, analyze predicted words using a novel semantic clustering technique, and discover insights using interactive visualizations. Collectively, these visualizations help users identify the likelihood and uniqueness of individual predictions, compare sets of predictions between prompts, and summarize patterns and relationships between predictions across all prompts. We demonstrate the capabilities of KnowledgeVis with feedback from six NLP experts as well as three different use cases: (1) probing biomedical knowledge in two domain-adapted models; and (2) evaluating harmful identity stereotypes and (3) discovering facts and relationships between three general-purpose models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑