固态物理启发的大语言模型压缩的预注册测试：小语言模型尺度上的聚类级阴性结果
Pre-registered tests of solid-state-physics-inspired LLM compression: a cluster-level negative result at small-language-model scale
浏览论文内容
中文总结 AI 辅助
本研究通过预注册和3σ门控的自主智能体程序，测试五种固态物理启发的LLM压缩映射，在≤350M尺度上多数假设被证伪，揭示注意力头呈临界态而非绝缘体，并贡献四项阴性结果及完整数据。
中文摘要 AI 辅助
我们报告了一个为期三个月的自主研究智能体程序，该程序在预训练语言模型上测试了五种受固态物理启发的压缩映射，所有预测在获取任何试点数据之前已提交至git，并设置3σ门控以决定通过（PASS）或搁置（SHELVE）。共同的理论锚点——单粒子密度矩阵的面积律/科恩近邻衰减——具有距离面（P001 Wannier，P002紧束缚）和秩面（P003 DMRG截断MLP，P005 Wilson-RG，P011张量列嵌入）。P005在第一阶段被抢先终止；四个第三阶段试点中有三个被证伪。在注意力面上，GPT-2-medium的注意力与距离关系在排除探针填充后，16个中位层头中有12个最佳拟合为拉伸指数，而紧束缚截断导致困惑度增加+96%（P002）；在Pythia-160M上，Wannier稀疏度0.054±0.004与PCA、随机Haar和恒等基线无法区分（P001）。在秩面上，每令牌张量列键维度不追踪惊奇度（r=0.016，而预注册值为0.65），且该格式导致膨胀而非压缩（P011）。P003结果混合：其缩放声明被搁置（r=-0.434），其MPO前提在阶段0失败，其跨论文检验最初为r=0.523，但在相同修正下崩溃至0.047，使得两项跨论文检验均为零。结果反转了预注册预测——大多数注意力头表现为科恩近邻绝缘体——而指向临界、玻璃态或重尾态；该反转特定于所测试的≤350M尺度，而秩面无增益结果在7-8B尺度上成立。我们贡献了预注册+3σ+聚类框架+仅追加目录纪律——包括为何我们自己的执行门被设计但未部署——四项预注册阴性结果及完整数据发布，以及上述反转。目录包含十八项已结束研究，其中十七项为阴性。
英文摘要
We report a three-month autonomous research-agent program testing five solid-state-physics-inspired compression mappings on pretrained language models, with predictions committed to git before any pilot data and a 3-sigma gate deciding PASS or SHELVE. The common anchor -- area-law / Kohn-nearsighted decay of the one-particle density matrix -- has a distance face (P001 Wannier, P002 tight-binding) and a rank face (P003 DMRG-truncated MLPs, P005 Wilson-RG, P011 tensor-train embeddings). P005 was pre-empted at Phase 1; three of four Phase-3 pilots were falsified. On the attention face, GPT-2-medium attention-versus-distance is best fit by a stretched exponential in 12 of 16 median-layer heads once probe padding is excluded, and a tight-binding cutoff costs +96% perplexity (P002); on Pythia-160M the Wannier sparsity 0.054 +/- 0.004 is indistinguishable from PCA, random-Haar and identity baselines (P001). On the rank face, per-token tensor-train bond dimension does not track surprisal (r = 0.016 vs a pre-registered 0.65) and the format inflates rather than compresses (P011). P003 is mixed: its scaling claim shelved (r = -0.434), its MPO premise died at stage-0, and its cross-paper check, r = 0.523 as first written, collapses to 0.047 under the same correction, leaving both cross-paper checks null. The results invert the pre-registered prediction that most attention heads behave like Kohn-nearsighted insulators, pointing instead to critical, glassy or heavy-tailed regimes; the inversion is specific to the <= 350M scale tested, while the rank-face no-gain result held to 7-8B. We contribute the pre-registration + 3-sigma + cluster-framing + append-only-catalogue discipline -- including why our own enforcement gate was designed but not deployed -- four pre-registered negative results with full data release, and the inversion. The catalogue holds eighteen concluded studies, seventeen negative.
发表机构
- University of Puerto Rico（波多黎各大学）
机构由 AI 辅助整理，请以论文原文为准。
