从少量样本中学习:多尺度下具有组合和局部性的变换不变支持向量机
Learning from Few Samples: Transformation-Invariant SVMs with Composition and Locality at Multiple Scales
摘要:
受小样本量学习问题的启发,本文展示了如何将使卷积神经网络(CNN)成功的那些属性纳入支持向量机(SVM)。特别重要的是能够结合不变性的领域知识,例如,图像的平移不变性基于一组变换上的\textit{maximum}相似性的核通常不是正定的。也许正是由于这个原因,它们没有被理论上研究。我们解决了这个空白,并表明,正定性确实持有\textit{with high probability}的核心的基础上,在小训练样本集制度的利益,最大的相似性,他们确实产生最好的结果,在该制度。我们还展示了如何额外的属性,如他们的能力,将在多个空间尺度的本地功能,例如,如在CNN中通过最大池化所做的那样,并且通过多层架构提供组合的好处,也可以嵌入到SVM中。我们通过广泛使用的图像集上的实验验证,所得到的支持向量机确实提供了优越的精度相比,建立良好的深神经网络基准小样本量。
英文摘要:
Motivated by the problem of learning with small sample sizes, this paper shows how to incorporate into support-vector machines (SVMs) those properties that have made convolutional neural networks (CNNs) successful. Particularly important is the ability to incorporate domain knowledge of invariances, e.g., translational invariance of images. Kernels based on the \textit{maximum} similarity over a group of transformations are not generally positive definite. Perhaps it is for this reason that they have not been studied theoretically. We address this lacuna and show that positive definiteness indeed holds \textit{with high probability} for kernels based on the maximum similarity in the small training sample set regime of interest, and that they do yield the best results in that regime. We also show how additional properties such as their ability to incorporate local features at multiple spatial scales, e.g., as done in CNNs through max pooling, and to provide the benefits of composition through the architecture of multiple layers, can also be embedded into SVMs. We verify through experiments on widely available image sets that the resulting SVMs do provide superior accuracy in comparison to well-established deep neural network benchmarks for small sample sizes.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。