具有两个非线性层的神经网络的多项式时间学习
Learning Neural Networks with Two Nonlinear Layers in Polynomial Time
摘要：
我们给出了一个学习神经网络的多项式时间算法，其中一层S形函数馈送到任何Lipschitz，单调激活函数（例如，S形或ReLU）。我们没有对网络的结构做任何假设，并且该算法成功地在$n$维的单位球上进行{ \em any}分布（隐藏的权重向量也具有单位范数）。这是学习具有两个非线性层的神经网络的第一个无约束、可证明有效的算法。我们的算法- { \em Alphatron}-是一个简单的，迭代更新规则，结合保序回归与核方法。它输出一个假设，产生有效的甲骨文访问可解释的功能。它还提出了一种新的方法，布尔学习问题，通过实值条件均值函数，回避传统的硬度结果从计算学习理论。沿着这些路线，我们继承和改进了许多长期的PAC学习布尔函数的结果，以更一般的，实值设置的{ \em概率概念}，一个模型，（不像PAC学习）需要非i.i.d.噪声容忍度
英文摘要：
We give a polynomial-time algorithm for learning neural networks with one layer of sigmoids feeding into any Lipschitz, monotone activation function (e.g., sigmoid or ReLU). We make no assumptions on the structure of the network, and the algorithm succeeds with respect to {\em any} distribution on the unit ball in $n$ dimensions (hidden weight vectors also have unit norm). This is the first assumption-free, provably efficient algorithm for learning neural networks with two nonlinear layers. Our algorithm-- {\em Alphatron}-- is a simple, iterative update rule that combines isotonic regression with kernel methods. It outputs a hypothesis that yields efficient oracle access to interpretable features. It also suggests a new approach to Boolean learning problems via real-valued conditional-mean functions, sidestepping traditional hardness results from computational learning theory. Along these lines, we subsume and improve many longstanding results for PAC learning Boolean functions to the more general, real-valued setting of {\em probabilistic concepts}, a model that (unlike PAC learning) requires non-i.i.d. noise-tolerance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
