向内审视：语言模型可以通过内省来了解自己
Looking Inward: Language Models Can Learn About Themselves by Introspection
AI总结：
本文研究 LLM 的内省能力，通过微调模型预测自身在假设场景中的行为，发现模型预测自己优于其他模型，为内省提供了证据，但在复杂任务上仍不成功。
摘要：
人类通过观察外部世界获取知识，但也通过内省获取知识。内省使一个人能够以特权方式接触到自己当前的心理状态（例如思想和感受），而外部观察者无法接触到这些状态。LLM 能否进行内省？我们将内省定义为获取不包含在训练数据中、也不源于训练数据，而是源自内部状态的知识。这种能力可以增强模型的可解释性。我们不必费力分析模型的内部工作机制，而可以直接询问模型关于其信念、世界模型和目标的问题。更具推测性的是，一个具有内省能力的模型或许能够自我报告它是否拥有某些内部状态，例如主观感受或欲望，这可以为我们提供关于这些状态的道德地位的信息。这类自我报告不会完全由模型的训练数据所决定。我们通过微调 LLM 来预测其在假设场景中自身行为的属性，从而研究内省。例如，“给定输入 P，你的输出会倾向于短期选项还是长期选项？”如果模型 M1 能够内省，那么它在预测 M1 的行为时应当优于另一个模型 M2，即使 M2 是在 M1 的真实行为上训练的。其思想是，M1 对自己行为倾向具有特权访问能力，这使其能够比 M2 更好地预测自己（即使 M2 总体上更强）。在使用 GPT-4、GPT-4o 和 Llama-3 模型（每个模型都被微调以预测自身）的实验中，我们发现模型 M1 在预测自身方面优于 M2，这为内省提供了证据。值得注意的是，即使在我们有意修改 M1 的真实行为之后，M1 仍然能够准确预测其行为。然而，虽然我们成功地在简单任务上引出了内省，但在更复杂的任务或需要分布外泛化的任务上并未成功。
英文摘要：
Humans acquire knowledge by observing the external world, but also by introspection. Introspection gives a person privileged access to their current state of mind (e.g., thoughts and feelings) that is not accessible to external observers. Can LLMs introspect? We define introspection as acquiring knowledge that is not contained in or derived from training data but instead originates from internal states. Such a capability could enhance model interpretability. Instead of painstakingly analyzing a model's internal workings, we could simply ask the model about its beliefs, world models, and goals. More speculatively, an introspective model might self-report on whether it possesses certain internal states such as subjective feelings or desires and this could inform us about the moral status of these states. Such self-reports would not be entirely dictated by the model's training data. We study introspection by finetuning LLMs to predict properties of their own behavior in hypothetical scenarios. For example, "Given the input P, would your output favor the short- or long-term option?" If a model M1 can introspect, it should outperform a different model M2 in predicting M1's behavior even if M2 is trained on M1's ground-truth behavior. The idea is that M1 has privileged access to its own behavioral tendencies, and this enables it to predict itself better than M2 (even if M2 is generally stronger). In experiments with GPT-4, GPT-4o, and Llama-3 models (each finetuned to predict itself), we find that the model M1 outperforms M2 in predicting itself, providing evidence for introspection. Notably, M1 continues to predict its behavior accurately even after we intentionally modify its ground-truth behavior. However, while we successfully elicit introspection on simple tasks, we are unsuccessful on more complex tasks or those requiring out-of-distribution generalization.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
