arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2305.18449cs.AIcs.CLcs.LGcs.SYeess.SY
收藏

驯服AI机器人:大语言模型中神经状态的可控性

Taming AI Bots: Controllability of Neural States in Large Language Models

University of California, Los Angeles(加州大学洛杉矶分校);University of Pennsylvania(宾夕法尼亚大学)

Stefano Soatto, Paulo Tabuada, Pratik Chaudhari, Tian Yu Liu

AI总结:

本文通过形式化“意义”并刻画训练良好的大语言模型,证明AI机器人在意义空间内几乎必然可控,并给出可控性的充要条件及防御采样设计思路。

摘要:

我们探讨这样一个问题:智能体能否通过恰当选择提示词,将AI机器人控制到任意状态。为此,我们首先引入一个便于分析的“意义”的形式化定义。然后,我们通过当今大语言模型(LLMs)大体上满足的条件,刻画LLMs据称训练所用的“有意义数据”以及“训练良好的LLMs”。虽然训练良好的LLM构建的意义嵌入空间是欧几里得空间,但意义本身并不构成向量(线性)子空间,而是其中的一个商空间。接着,我们刻画了对于某些输入提示词,LLM状态能够到达的意义子集,并证明训练良好的机器人能够到达任意意义,尽管概率很小。随后,我们引入更强的可控性概念,即“几乎必然可达性”,并证明当限制在意义空间内时,AI机器人是可控的。为此,我们先引入注意力型AI机器人的函数刻画,最后推导出可控性的充分必要条件。AI机器人可控这一事实意味着对手可以将其引导至任意状态。然而,采样过程可以被设计用来对抗不良行为,并在越过状态空间边界之前避免到达不希望的区域。

英文摘要:

We tackle the question of whether an agent can, by suitable choice of prompts, control an AI bot to any state. To that end, we first introduce a formal definition of ``meaning'' that is amenable to analysis. Then, we characterize ``meaningful data'' on which large language models (LLMs) are ostensibly trained, and ``well-trained LLMs'' through conditions that are largely met by today's LLMs. While a well-trained LLM constructs an embedding space of meanings that is Euclidean, meanings themselves do not form a vector (linear) subspace, but rather a quotient space within. We then characterize the subset of meanings that can be reached by the state of the LLMs for some input prompt, and show that a well-trained bot can reach any meaning albeit with small probability. We then introduce a stronger notion of controllability as {\em almost certain reachability}, and show that, when restricted to the space of meanings, an AI bot is controllable. We do so after introducing a functional characterization of attentive AI bots, and finally derive necessary and sufficient conditions for controllability. The fact that AI bots are controllable means that an adversary could steer them towards any state. However, the sampling process can be designed to counteract adverse actions and avoid reaching undesirable regions of state space before their boundary is crossed.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑