发表机构
Ocean University of China（中国海洋大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
研究长上下文检索中FFN的作用，通过逐层缩放模型自身FFN写入进行测试，发现其响应面带符号、特定层且任务条件化，两种单调角色可由写入梯度对齐诊断，还能预测检索衰减损伤及改进检索边界。
AI 中文摘要
前馈神经网络（FFNs）常被视为参数化存储器。在长上下文检索中，关键问题不仅在于它们存储了什么，还在于其固有的残差写入是推动当前检索状态接近还是远离正确答案。我们通过逐层缩放模型自身的FFN写入来进行测试，不编辑权重或注入外部引导向量。在受控的文字和语义检索套件中，FFN的响应面是带符号的、特定层的且任务条件化的：在8个模型套件案例中的7个中，最终的FFN是抑制器，60%的层在检索模式之间切换角色。沿着固有写入的局部方向导数区分了两种单调角色。在安全过滤的LongBench检索问答探针上，相同的诊断预测了Qwen2.5 - 7B上原始R^2 = 0.796和Qwen3.5 - 9B上0.791的衰减损伤。这些结果表明，FFN缩放揭示了检索中带符号的、任务条件化的残差写入结构，并且写入梯度对齐是这两种单调角色的紧凑诊断。
英文摘要
FFNs are often treated as parametric memories. In long-context retrieval, however, the sharper question is not only what they store, but whether their native residual writes push the current retrieval state toward or away from the correct answer. We test this by scaling the model's own FFN write one layer at a time, without editing weights or injecting external steering vectors. Across controlled literal and semantic retrieval suites, native FFN response surfaces are signed, layer-specific, and task-conditioned: the final FFN is a suppressor in 7 of 8 model-suite cases, and 60% of layers switch role between retrieval modes (95% CI [50%, 69%]). A local directional derivative along the native write separates the two monotone roles: suppressors have negative derivative in 34/35 cases, and amplifiers have positive derivative in 18/18 cases, so the roles are not reducible to write size. On a safety-filtered LongBench retrieval-QA probe, the same diagnostic predicts attenuation damage with raw R^2=0.796 on Qwen2.5-7B and 0.791 on Qwen3.5-9B; a held-out suppressor-attenuation policy improves retrieval margins over random and norm-matched controls. These results show that native FFN scaling exposes a signed, task-conditioned residual-write structure in retrieval, and that write-gradient alignment is a compact diagnostic for the two monotone roles.
Comments23 pages, 8 figures, 12 tables
