arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.01702cs.CLcs.AI
收藏

语言模型的流畅梦境

Fluent dreaming for language models

Confirm Labs

T. Ben Thompson, Zygimantas Straznickas, Michael Sklar

AI总结:

本文提出进化提示优化算法,实现语言模型的流畅梦境,通过优化提示同时最大化内部特征与流畅性,自动探索模型对分布外提示的内部反应。

摘要:

特征可视化,也称为“梦境”,通过优化输入以最大化神经元激活或其他内部组件,为视觉模型提供洞察。然而,由于输入空间是离散的,梦境尚未成功应用于语言模型。我们扩展了语言模型对抗攻击文献中的贪婪坐标梯度方法,设计了进化提示优化(EPO)算法。EPO优化输入提示,以同时最大化选定内部特征与提示流畅性之间的帕累托前沿,从而实现语言模型的流畅梦境。我们展示了神经元、输出逻辑和激活空间中任意方向的梦境。我们测量了所得提示的流畅性,并将语言模型梦境与最大激活数据集示例进行比较。关键的是,流畅梦境允许自动探索模型内部对轻度分布外提示的反应行为。运行EPO的代码可在https://github.com/Confirm-Solutions/dreamy获取。展示代码使用的配套页面位于https://confirmlabs.org/posts/dreamy.html。

英文摘要:

Feature visualization, also known as "dreaming", offers insights into vision models by optimizing the inputs to maximize a neuron's activation or other internal component. However, dreaming has not been successfully applied to language models because the input space is discrete. We extend Greedy Coordinate Gradient, a method from the language model adversarial attack literature, to design the Evolutionary Prompt Optimization (EPO) algorithm. EPO optimizes the input prompt to simultaneously maximize the Pareto frontier between a chosen internal feature and prompt fluency, enabling fluent dreaming for language models. We demonstrate dreaming with neurons, output logits and arbitrary directions in activation space. We measure the fluency of the resulting prompts and compare language model dreaming with max-activating dataset examples. Critically, fluent dreaming allows automatically exploring the behavior of model internals in reaction to mildly out-of-distribution prompts. Code for running EPO is available at https://github.com/Confirm-Solutions/dreamy. A companion page demonstrating code usage is at https://confirmlabs.org/posts/dreamy.html

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑