通过对极分离的密码分析提取孤立无偏GLU前馈块
Cryptanalytic Extraction of Isolated Bias-Free GLU Feed-Forward Blocks by Antipodal Separation
浏览论文内容
中文总结 AI 辅助
该研究针对现有密码分析提取方法无法恢复现代语言模型的无偏GLU前馈块的问题，提出多阶段前向查询恢复原语，在多种模型上实现亚百分位或低误差，且明确其非端到端攻击的性质。
中文摘要 AI 辅助
密码分析提取已在ReLU网络、使用GELU或SiLU等逐元素激活函数的网络，以及Transformer的最终投影矩阵上得到验证。这些方法无法恢复许多现代语言模型中使用的无偏门控线性单元（GLU）前馈块，这类块在每个隐藏单元内将激活后的线性投影与第二个学习到的线性投影相乘，是一种双分支结构，不在上述方法所针对的网络类别和最终层设置中。我们提出一种用于孤立无偏GLU块的构造性多阶段前向查询恢复原语：有限差分曲率提供门方向候选，通过x和-x处的配对观测值分离门的幅度、方向和值分支耦合。在高精度目标上，6层Qwen、8192单元的Llama子问题以及全维度Gemma块均达到亚百分位中位数验证误差；4种有限精度配置的中位数误差仍低于5%，但无法复现所有存储权重。这些孤立块实验并非端到端模型API攻击，从最终模型输出推导所需的内部块响应仍是未解决的问题。
英文摘要
Cryptanalytic extraction has been demonstrated for ReLU networks, for networks using componentwise activations such as GELU or SiLU, and for a Transformer's final projection matrix. These methods do not recover the bias-free Gated Linear Unit (GLU) feed-forward blocks used in many modern language models. Such a block multiplies an activated linear projection by a second learned linear projection within each hidden unit, a two-branch structure absent from the network classes and final-layer setting addressed by those methods. We give a constructive, multi-stage forward-query recovery primitive for isolated bias-free GLU blocks. Finite-difference curvature supplies gate-direction candidates, and paired observations at x and -x separate gate magnitude, orientation, and value-branch coupling. Across high-precision targets, six Qwen layers, an 8,192-unit Llama subproblem, and a full-dimensional Gemma block all reach sub-percent median validation error. Four finite-precision configurations remain below 5 percent median error, but none reproduces every stored weight. These isolated-block experiments are not an end-to-end model-API attack: deriving the required internal block responses from final model outputs remains unsolved.
