大规模可解释性：识别 Alpaca 中的因果机制
Interpretability at Scale: Identifying Causal Mechanisms in Alpaca
AI总结：
本文提出 Boundless DAS 方法，将可解释性扩展到大型语言模型，成功在 Alpaca 中识别出两个布尔变量构成的因果机制，且对齐对输入和指令变化稳健，为理解大模型内部运作迈出第一步。
摘要：
获得大型通用语言模型的人类可解释解释是 AI 安全的一个紧迫目标。然而，同样重要的是，我们的可解释性方法要忠实于模型行为背后的因果动态，并且能够稳健地泛化到未见过的输入。分布式对齐搜索（DAS）是一种强大的梯度下降方法，基于因果抽象理论，已发现可解释的符号算法与针对特定任务微调的小型深度学习模型之间的完美对齐。在本文中，我们通过用学习参数替换剩余的暴力搜索步骤，显著扩展了 DAS——我们称之为无界 DAS（Boundless DAS）的方法。这使我们能够在大型语言模型遵循指令时高效地搜索其可解释的因果结构。我们将 Boundless DAS 应用于 Alpaca 模型（7B 参数），该模型开箱即用地解决了一个简单的数值推理问题。通过 Boundless DAS，我们发现 Alpaca 通过实现一个具有两个可解释布尔变量的因果模型来解决这个问题。此外，我们发现神经表示与这些变量的对齐对输入和指令的变化具有稳健性。这些发现标志着朝着忠实理解我们不断增长且部署最广泛的语言模型内部运作迈出了第一步。我们的工具可扩展到更大的 LLM，并已在 `https://github.com/stanfordnlp/pyvene` 公开发布。
英文摘要：
Obtaining human-interpretable explanations of large, general-purpose language models is an urgent goal for AI safety. However, it is just as important that our interpretability methods are faithful to the causal dynamics underlying model behavior and able to robustly generalize to unseen inputs. Distributed Alignment Search (DAS) is a powerful gradient descent method grounded in a theory of causal abstraction that has uncovered perfect alignments between interpretable symbolic algorithms and small deep learning models fine-tuned for specific tasks. In the present paper, we scale DAS significantly by replacing the remaining brute-force search steps with learned parameters -- an approach we call Boundless DAS. This enables us to efficiently search for interpretable causal structure in large language models while they follow instructions. We apply Boundless DAS to the Alpaca model (7B parameters), which, off the shelf, solves a simple numerical reasoning problem. With Boundless DAS, we discover that Alpaca does this by implementing a causal model with two interpretable boolean variables. Furthermore, we find that the alignment of neural representations with these variables is robust to changes in inputs and instructions. These findings mark a first step toward faithfully understanding the inner-workings of our ever-growing and most widely deployed language models. Our tool is extensible to larger LLMs and is released publicly at `https://github.com/stanfordnlp/pyvene`.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
