arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2403.03028cs.AIcs.CL
收藏

单词重要性解释提示词如何影响语言模型输出

Word Importance Explains How Prompts Affect Language Model Outputs

DataRobot Germany(DataRobot德国);DataRobot

Stefan Hackmann, Haniyeh Mahmoudian, Mark Steadman, Michael Schmidt

AI总结:

本文提出通过掩码系统提示词中的单词并聚合多输入文本评分来计算单词重要性,以解释提示词对LLM输出的影响,实验显示其与后缀预期重要性高度一致。

摘要:

大型语言模型(LLM)的出现彻底改变了各行各业的众多应用。然而,其“黑箱”特性往往阻碍人们理解它们如何作出特定决策,引发了对其透明度、可靠性和伦理使用的担忧。本研究提出一种提升LLM可解释性的方法:通过改变提示词中的单个单词,揭示其对模型输出的统计影响。该方法受表格数据置换重要性启发,对系统提示词中的每个单词进行掩码,并基于在多个用户输入上聚合的可用文本评分来评估其对输出的影响。与传统注意力不同,单词重要性衡量提示词单词对任意定义的文本评分的影响,从而能够将单词重要性分解为所关注的具体度量,包括偏见、阅读难度、冗长程度等。该过程还能在注意力权重不可用时衡量影响。为检验该方法的保真度,我们探究了向多个不同系统提示词添加不同后缀的效果,并比较不同大型语言模型随后生成的内容。结果表明,对于多种评分函数,单词重要性分数与预期后缀重要性密切相关。

英文摘要:

The emergence of large language models (LLMs) has revolutionized numerous applications across industries. However, their "black box" nature often hinders the understanding of how they make specific decisions, raising concerns about their transparency, reliability, and ethical use. This study presents a method to improve the explainability of LLMs by varying individual words in prompts to uncover their statistical impact on the model outputs. This approach, inspired by permutation importance for tabular data, masks each word in the system prompt and evaluates its effect on the outputs based on the available text scores aggregated over multiple user inputs. Unlike classical attention, word importance measures the impact of prompt words on arbitrarily-defined text scores, which enables decomposing the importance of words into the specific measures of interest--including bias, reading level, verbosity, etc. This procedure also enables measuring impact when attention weights are not available. To test the fidelity of this approach, we explore the effect of adding different suffixes to multiple different system prompts and comparing subsequent generations with different large language models. Results show that word importance scores are closely related to the expected suffix importances for multiple scoring functions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑