发表机构
Sandia National Laboratories; New Mexico Institute of Mining and Technology; Institute for Complex Additive Systems Analysis（桑迪亚国家实验室; 新墨西哥矿业理工大学; 复杂增材系统分析研究所）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文提出流递归模型（SRM），通过多潜在流递归细化暴露内部计算结构，在每参数性能上与GPT-2相当，为可扩展机制可解释性提供实用架构基础。
AI 中文摘要
机制可解释性旨在对大型语言模型（LLM）的内部行为做出可验证的陈述。许多可解释性技术难以随着架构规模和深度的增加而扩展。我们的解决方案是引入结构上更利于可解释性的较小模型。在这项工作中，我们引入了流递归模型（SRM），这是对层级推理模型（HRM）的一种改进，旨在暴露内部计算结构，同时保持可扩展性。SRM将计算组织为多个相互作用的潜在流，这些流通过递归细化进行更新，从而能够直接分析流动态、因果贡献和路由行为。SRM在每参数基础上实现了与GPT-2相当的性能。我们的分析揭示了各流之间一致且独特的行为，表明存在结构化的专门化和交互。这些结果表明，SRM为可扩展的机制可解释性提供了一个实用的架构基础，并为未来在推理性能和可解释性方面的研究开辟了有前景的途径。
英文摘要
Mechanistic interpretability seeks to make verifiable statements about the internal behavior of large language models (LLMs). Many interpretability techniques struggle to scale with the increasing size and depth of architectures. Our solution to this is to introduce smaller models with structures that lend themselves to interpretability. In this work, we introduce the Stream Recursion Model (SRM), a modification of the Hierarchical Reasoning Model (HRM) designed to expose internal computational structure while remaining scalable. SRM organizes computation into multiple interacting latent streams that are updated through recursive refinement, enabling direct analysis of stream dynamics, causal contribution, and routing behavior. SRM achieves performance comparable to GPT-2 on a per-parameter basis. Our analysis reveals consistent and distinct behavior across streams, indicating structured specialization and interaction. These results suggest that SRM provides a practical architectural foundation for scalable mechanistic interpretability and opens up promising avenues for future research in both reasoning performance and interpretability.
Comments21 pages, 27 figures
