脉冲驱动神经架构:用于鲁棒连续时间序列处理的可学习振荡动力学
Pulse-Driven Neural Architecture: Learnable Oscillatory Dynamics for Robust Continuous-Time Sequence Processing
AI总结:
PDNA通过引入可学习的振荡动力学,提升连续时间序列处理的鲁棒性,尤其在输入中断情况下表现优异。
摘要:
我们引入PDNA(脉冲驱动神经架构),一种通过可学习的振荡动力学增强连续时间递归网络的方法,该动力学能够独立于外部输入维持内部状态的演变。基于闭式连续时间(CfC)网络,PDNA添加了两个组件:(1)一个脉冲模块,生成结构化的振荡$A \cdot \sin(ωt + φ(h))$,具有可学习的频率和状态依赖的相位;(2)一个自注意力模块,对隐藏状态应用递归自注意力。通过在具有五个随机种子的顺序MNIST(sMNIST)上进行受控的消融研究,我们评估了间隙鲁棒性——在测试时去除输入序列部分时保持性能的能力。我们的关键发现是,结构化的振荡动力学显著提高了对输入中断的鲁棒性:自注意力变体在基线模型上实现了统计显著的2.78个百分点的多间隙优势(p=0.041),而脉冲变体显示出4.62个百分点的优势,具有较大的效应量(Cohen's d=0.87)。噪声控制(等量的随机扰动)未提供任何好处,证实了优势是结构上的而非仅仅是动态的。这些结果提供了证据,表明连续时间模型可以受益于生物启发的内部振荡机制以提高时间鲁棒性。
英文摘要:
We introduce PDNA (Pulse-Driven Neural Architecture), a method for augmenting continuous-time recurrent networks with learnable oscillatory dynamics that maintain internal state evolution independently of external input. Built on Closed-form Continuous-time (CfC) networks, PDNA adds two components: (1) a pulse module that generates structured oscillations $A \cdot \sin(ωt + φ(h))$ with learnable frequencies and state-dependent phase, and (2) a self-attend module that applies recurrent self-attention to the hidden state. Through a controlled ablation study on sequential MNIST (sMNIST) with five random seeds, we evaluate gap robustness -- the ability to maintain performance when portions of the input sequence are removed at test time. Our key finding is that structured oscillatory dynamics significantly improve robustness to input interruptions: the self-attend variant achieves a statistically significant 2.78 percentage point multi-gap advantage over baseline ($p = 0.041$), while the pulse variant shows a 4.62 pp advantage with large effect size (Cohen's $d = 0.87$). A noise control (random perturbation of equal magnitude) provides no benefit, confirming that the advantage is structural rather than merely dynamic. These results provide evidence that continuous-time models can benefit from biologically-inspired internal oscillatory mechanisms for temporal robustness.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。