循环表示的有限滞后算子几何
Auditing Finite-Lag Statistics of Recurrent Representations
AI总结:
提出有限滞后算子几何,通过源-后继对估计条件传输律,导出传输张量和循环量,证明仿射协方差与分离性,揭示循环网络架构差异。
摘要:
循环表示是轨迹,但表示几何通常从静态快照测量。我们从观测到的源-后继对 $(X_t,X_{t+\Delta})$ 为循环隐藏状态开发了有限滞后算子几何。原始量是条件传输律 $Q_\Delta(dy\mid x)$,通过密集高斯源平滑算子估计。从该有向有限滞后律,我们导出一个源中心传输张量 $G_\Delta$,它精确分解为条件扩散和相干位移,以及一个反对称坐标循环 $W_\Delta^\rho$,它总结了有向滞后流。我们证明了标量汇总的仿射协方差与显式度量依赖、有界轨迹云上的密集估计器稳定性,以及一个有限滞后分离结果,表明源中心传输检测到无穷小 carre-du-champ 几何未记录的确定性循环运动。一个线性-高斯闭式根据更新 $A_\Delta$、源协方差和创新协方差校准这些量。受控实验验证了分解、循环、协方差和稳定性预测。在性能匹配的重复复制网络中,该框架揭示了总传输尺度和相干位移迹上的架构依赖差异,而相干位移分数依赖于度量和分辨率。
英文摘要:
Hidden-state trajectories are computational traces that record how recurrent representations evolve as networks process sequences. When exact state repeats are unavailable, one way to estimate conditional motion is to borrow successors from nearby states, yielding a finite-bandwidth successor smoother. However, this procedure can change the displacement being measured. At a median-distance bandwidth, write-phase displacement energy is about $1.88\times$ the exact value for GRU and LSTM. Elman instead shows near agreement in total energy despite large, opposing finite-bandwidth contributions. We therefore compute the conditional-mean energy fraction exactly on repeat-copy. Training reverses the architecture ordering of this fraction, with GRU and LSTM rising to about $94\%$ while Elman falls to about $55\%$. The centered remainder is small in energy but can still expose input information. For LSTM, linear four-bit decoding rises from $0.804$ on total displacement to $1.000$ after exact conditional centering.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。