深层网络具有封闭形式的权重
Deep Layer-wise Networks Have Closed-Form Weights
摘要:
目前,神经科学界对大脑执行反向传播(BP)的可能性存在争议。为了更好地模拟大脑,已经提出了一种仅使用“单次向前传递”来训练网络\textit{one layer at a time}的方法,作为绕过BP的替代方案;我们将这些网络称为“逐层”网络。我们通过回答两个悬而未决的问题来继续分层网络的工作。第一,$\textit{do they have a closed-form solution?}$第二,$\textit{how do we know when to stop adding more layers?}$这项工作证明了核均值嵌入是实现网络全局最优的封闭形式权重,同时驱动这些网络收敛到一个高度理想的分类核;我们称之为$\textit{Neural Indicator Kernel}$。
英文摘要:
There is currently a debate within the neuroscience community over the likelihood of the brain performing backpropagation (BP). To better mimic the brain, training a network \textit{one layer at a time} with only a "single forward pass" has been proposed as an alternative to bypass BP; we refer to these networks as "layer-wise" networks. We continue the work on layer-wise networks by answering two outstanding questions. First, $\textit{do they have a closed-form solution?}$ Second, $\textit{how do we know when to stop adding more layers?}$ This work proves that the Kernel Mean Embedding is the closed-form weight that achieves the network global optimum while driving these networks to converge towards a highly desirable kernel for classification; we call it the $\textit{Neural Indicator Kernel}$.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。