我、我自己与 AI:面向 LLM 的情境觉知数据集(SAD)
Me, Myself, and AI: The Situational Awareness Dataset (SAD) for LLMs
AI总结:
该研究提出包含 7 类任务、逾 1.3 万个问题的 SAD 基准,通过问答与指令遵循测试量化 LLM 的自我认知与情境觉知,并评估 16 个模型。
摘要:
诸如 ChatGPT 之类的 AI 助手经过训练,会在回应用户时说“我是一个大型语言模型”。这引发了一些问题:这类模型是否知道自己是 LLM,并可靠地依据这一知识行动?它们是否意识到自己当前所处的情境,例如已被公开展开部署?我们将模型对自身及其情境的知识称为情境觉知。为了量化 LLM 中的情境觉知,我们引入了一系列基于问答和指令遵循的行为测试。这些测试构成了 Situational Awareness Dataset(SAD),这是一个包含 7 个任务类别和超过 13,000 个问题的基准。该基准测试多项能力,包括 LLM:(i)识别其自身生成文本的能力,(ii)预测其自身行为的能力,(iii)判断一个提示来自内部评估还是真实世界部署的能力,以及(iv)遵循依赖于自我知识的指令的能力。我们在 SAD 上评估了 16 个 LLM,包括基座(预训练)模型和聊天模型。尽管所有模型的表现都优于随机水平,但即使是得分最高的模型(Claude 3 Opus)在某些任务上也远未达到人类基线。我们还观察到,SAD 上的表现只能由通用知识指标(例如 MMLU)部分预测。经过微调以充当 AI 助手的聊天模型在 SAD 上优于其对应的基座模型,但在通用知识任务上并非如此。SAD 的目的是通过将 LLM 的情境觉知分解为可量化的能力,促进对其的科学理解。情境觉知之所以重要,是因为它增强了模型自主规划和行动的能力。虽然这对自动化具有潜在益处,但也带来了与 AI 安全和控制相关的新型风险。代码和最新结果可在 https://situational-awareness-dataset.org 获取。
英文摘要:
AI assistants such as ChatGPT are trained to respond to users by saying, "I am a large language model". This raises questions. Do such models know that they are LLMs and reliably act on this knowledge? Are they aware of their current circumstances, such as being deployed to the public? We refer to a model's knowledge of itself and its circumstances as situational awareness. To quantify situational awareness in LLMs, we introduce a range of behavioral tests, based on question answering and instruction following. These tests form the $\textbf{Situational Awareness Dataset (SAD)}$, a benchmark comprising 7 task categories and over 13,000 questions. The benchmark tests numerous abilities, including the capacity of LLMs to (i) recognize their own generated text, (ii) predict their own behavior, (iii) determine whether a prompt is from internal evaluation or real-world deployment, and (iv) follow instructions that depend on self-knowledge. We evaluate 16 LLMs on SAD, including both base (pretrained) and chat models. While all models perform better than chance, even the highest-scoring model (Claude 3 Opus) is far from a human baseline on certain tasks. We also observe that performance on SAD is only partially predicted by metrics of general knowledge (e.g. MMLU). Chat models, which are finetuned to serve as AI assistants, outperform their corresponding base models on SAD but not on general knowledge tasks. The purpose of SAD is to facilitate scientific understanding of situational awareness in LLMs by breaking it down into quantitative abilities. Situational awareness is important because it enhances a model's capacity for autonomous planning and action. While this has potential benefits for automation, it also introduces novel risks related to AI safety and control. Code and latest results available at https://situational-awareness-dataset.org .
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。