arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1808.04475stat.MLcs.LG
收藏

内核流:从数据到深渊的学习内核

Kernel Flows: from learning kernels from data into the abyss

California Institute of Technology(加州理工学院)

Houman Owhadi, Gene Ryan Yoo

摘要:

学习可以被看作是通过对训练数据进行插值来逼近未知函数。克里金法提供了一个解决这个问题的基础上,事先规范的内核。我们探讨了一个数值近似方法内核的选择/建设的基础上的简单的前提下,内核必须是好的,如果插值点的数量可以减半,而没有显着损失的准确性(测量使用固有的RKHS规范$\|\cdot\|$与内核)。我们首先测试和激励这个想法上一个简单的问题,恢复的格林函数的椭圆偏微分方程(非齐次系数)从稀疏观察其解决方案之一。接下来,我们考虑学习具有$F_{n+1}=(I_d+εG_{n+1})\circ F_n$和$G_{n+1} \in \operatorname{Span}\{K_1(F_n(x_i),\cdot)\}$的$K_1(F_n(x),F_n(x'))$形式的深度内核的非参数族的问题。使用所提出的方法,构建内核变得等同于集成随机数据驱动的动态系统,这允许训练非常深(无底)的网络并探索其属性。这些网络通过增量数据相关变形/扰动在内核和输入空间中构建流图来学习(作为对抗性例子的合作对应物出现),在深度上,它们(1)可以从每个类的一个数据点实现准确的分类(2)似乎可以学习每个类的原型(3)扩大不同类中的点之间的距离,缩小同一类中的点之间的距离。对于由卷积神经网络的权重参数化的内核,最小化通过将插值点的随机子集减半而引起的近似误差似乎优于具有相对熵和辍学的训练(相同的CNN架构)。

英文摘要:

Learning can be seen as approximating an unknown function by interpolating the training data. Kriging offers a solution to this problem based on the prior specification of a kernel. We explore a numerical approximation approach to kernel selection/construction based on the simple premise that a kernel must be good if the number of interpolation points can be halved without significant loss in accuracy (measured using the intrinsic RKHS norm $\|\cdot\|$ associated with the kernel). We first test and motivate this idea on a simple problem of recovering the Green's function of an elliptic PDE (with inhomogeneous coefficients) from the sparse observation of one of its solutions. Next we consider the problem of learning non-parametric families of deep kernels of the form $K_1(F_n(x),F_n(x'))$ with $F_{n+1}=(I_d+εG_{n+1})\circ F_n$ and $G_{n+1} \in \operatorname{Span}\{K_1(F_n(x_i),\cdot)\}$. With the proposed approach constructing the kernel becomes equivalent to integrating a stochastic data driven dynamical system, which allows for the training of very deep (bottomless) networks and the exploration of their properties. These networks learn by constructing flow maps in the kernel and input spaces via incremental data-dependent deformations/perturbations (appearing as the cooperative counterpart of adversarial examples) and, at profound depths, they (1) can achieve accurate classification from only one data point per class (2) appear to learn archetypes of each class (3) expand distances between points that are in different classes and contract distances between points in the same class. For kernels parameterized by the weights of Convolutional Neural Networks, minimizing approximation errors incurred by halving random subsets of interpolation points, appears to outperform training (the same CNN architecture) with relative entropy and dropout.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑