受生物学启发的联合分布神经元：基于层次相关性重建的多向传播神经元
Biology-inspired joint distribution neurons based on Hierarchical Correlation Reconstruction allowing for multidirectional propagation of values and densities
AI总结：
本文提出了一种受生物学启发的联合分布神经元，通过层次相关性重建实现多向值和密度传播，改进了传统人工神经元在学习、灵活性和鲁棒性方面的不足。
摘要：
最近，一百万生物神经元（BNN）在现代强化学习（RL）方法中表现出色，尤其在Pong游戏中表现优异，表明它们在学习、灵活性和鲁棒性方面仍具有显著优势，提示需要改进当前人工神经元（如MLP/KAN）以更好地与生物神经元一致。本文提出了一种扩展KAN方法的神经元，包含局部联合分布模型：ρ(x)=∑_{j∈B} a_j f_j(x)对于x∈[0,1]^d，增加了对KAN的解释和信息流控制，并允许逐步补充生物神经元的三个基本特性：1）生物轴突可以双向传播，而当前人工神经元仅单向传播，联合分布神经元可通过替换变量获得条件值/分布；2）动物表现出风险规避，需要处理方差，现实世界更需要概率模型，所提方法可预测和传播分布作为矩向量（期望值、方差等）；3）生物神经元需要局部训练，除了反向传播外，所提方法还允许其他训练方式，如直接训练、张量分解或最终的局部和有前景的信息瓶颈。所提方法非常通用，也可用于扩展softmax在transformer或JEPA嵌入中的应用，暗示特征是现实世界属性联合密度的混合矩。
英文摘要：
Recently a million of biological neurons (BNN) has turned out better from modern RL methods in playing Pong~\cite{RL}, reminding they are still qualitatively superior e.g. in learning, flexibility and robustness - suggesting to try to improve current artificial e.g. MLP/KAN for better agreement with biological. There is proposed extension of KAN approach to neurons containing model of local joint distribution: $ρ(\mathbf{x})=\sum_{\mathbf{j}\in B} a_\mathbf{j} f_\mathbf{j}(\mathbf{x})$ for $\mathbf{x} \in [0,1]^d$, adding interpretation and information flow control to KAN, and allowing to gradually add missing 3 basic properties of biological: 1) biological axons propagate in both directions~\cite{axon}, while current artificial are focused on unidirectional propagation - joint distribution neurons can repair by substituting some variables to get conditional values/distributions for the remaining. 2) Animals show risk avoidance~\cite{risk} requiring to process variance, and generally real world rather needs probabilistic models - the proposed can predict and propagate also distributions as vectors of moments: (expected value, variance) or higher. 3) biological neurons require local training, and beside backpropagation, the proposed allows many additional ways, like direct training, through tensor decomposition, or finally local and promising: information bottleneck. Proposed approach is very general, can be also used as extension of softmax in embeddings of e.g. transformer, JEPA, Mamba, suggesting interpretation that features are mixed moments of joint density of real-world properties.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
