发表机构
Electronics and Telecommunications Research Institute（电子通信研究院）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本研究发现Transformer与Mamba架构虽微观机制不同，却存在相近的近边际慢模动力学，其集体红外组织具有普适性，为认知场理论提供了独立验证。
AI 中文摘要
不同神经架构是否会形成共同的集体动力学仍是一个开放问题。近期对Transformer语言模型的分析显示，其存在近乎平坦、弱红外增强的时间尺度态密度（TDOS），与近边际长程记忆动力学相关。本研究测试结构高度相近的Mamba（其选择性状态空间动力学具有完全不同的微观机制）是否也会出现此类组织。Mamba的弛豫动力学可在三个层面解析：学习到的状态空间生成器的本征谱、其输入条件依赖的选择性重标度、以及通过其雅可比矩阵测得的完整模块的集体TDOS。这些谱并不完全相同：选择性动力学与其余模块变换会显著重组微观弛豫层级。不过，完整模块会形成可重复的慢模连续体，其红外区域随序列长度增加而逐渐被更清晰地分辨。累积分析得到ρ(λ)~λ^β，长序列Mamba的指数稳定在β_M≃-0.17附近，对应的记忆动力学遵循K(t)~t^-(1+β)，接近边际1/t regime。尽管微观动力学根本不同，Transformer完整模块谱却表现出高度相近的红外组织，代表性指数约为β_Tr~-0.1。这些结果将显式状态空间记忆与集体红外组织区分开来，表明不同序列架构可形成高度相近的近边际慢模动力学，它们将红外集体组织扩展至Transformer之外，并为认知场理论描述的动力学结构提供独立验证。
英文摘要
Whether distinct neural architectures develop common collective dynamics remains an open question. Recent analysis of Transformer language models revealed a nearly flat, weakly infrared-enhanced time-scale density of states (TDOS) associated with near-marginal long-memory dynamics. Here we test whether a closely related organization emerges in Mamba, whose selective state-space dynamics provides a fundamentally different microscopic mechanism. Mamba allows relaxation dynamics to be resolved at three levels: the intrinsic spectrum of the learned state-space generator, its input-conditioned selective rescaling, and the collective TDOS of the complete block measured from its Jacobian. These spectra are not identical: selective dynamics and the remaining block transformations substantially reorganize the microscopic relaxation hierarchy. Nevertheless, the full block develops a reproducible slow-mode continuum whose infrared sector becomes progressively better resolved with increasing sequence length. Cumulative analysis yields $ρ(λ)\simλ^β$, with the long-sequence Mamba exponent stabilizing near $β_{\rm M}\simeq-0.17$. The corresponding memory dynamics follows $K(t)\sim t^{-(1+β)}$, close to the marginal $1/t$ regime. Despite fundamentally different microscopic dynamics, Transformer full-block spectra exhibit closely related infrared organization, with representative exponents of order $β_{\rm Tr}\sim-0.1$. These results separate explicit state-space memory from collective infrared organization and show that distinct sequence architectures can develop closely related near-marginal slow-mode dynamics. They extend infrared collective organization beyond Transformers and provide an independent test of the dynamical structure described by Cognitive Field Theory.
Comments31 pages, 12 figures
