arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1808.10078stat.MLcs.LG
收藏

相似性和群等变表示的判别学习

Discriminative Learning of Similarity and Group Equivariant Representations

Toyota Technological Institute at Chicago(芝加哥丰田技术研究所);The University of Chicago(芝加哥大学);Fermilab(费米实验室)

Shubhendu Trivedi

摘要:

机器学习中最基本的问题之一是比较示例:给定一对对象,我们希望返回一个值,该值表示(不)相似度。相似性通常是特定于任务的,预定义的距离可能表现不佳,导致度量学习的工作。然而,能够学习相似性敏感的距离函数也需要获得手头对象的丰富的、有区别的表示。在这篇论文中,我们提出了对这两个目的的贡献。在论文的第一部分中,假设数据具有良好的表示,我们提出了一个度量学习的公式,与以前的工作相比,它更直接地尝试优化k-NN的准确性。我们还提出了扩展这个公式的度量学习kNN回归,非对称相似性学习和判别学习的汉明距离。在第二部分中,我们考虑一种情况,即我们在有限的计算预算上,即在可能度量的空间上进行优化是不可行的,但是访问标签感知距离度量仍然是可取的。我们提出了一个简单的,计算成本低的方法来估计一个动机良好的度量,只依赖于梯度估计,讨论理论和实验结果。在最后一部分,我们考虑组等变卷积神经网络(GCNN)来解决代表性问题。对称变换的等方差在GCNN中被显式编码;经典的CNN是最简单的例子。特别是,我们提出了一个SO(3)-等变神经网络架构的球形数据,完全在傅立叶空间中运行,同时也提供了一个形式主义的设计完全傅立叶神经网络是等变的任何连续紧致群的行动。

英文摘要:

One of the most fundamental problems in machine learning is to compare examples: Given a pair of objects we want to return a value which indicates degree of (dis)similarity. Similarity is often task specific, and pre-defined distances can perform poorly, leading to work in metric learning. However, being able to learn a similarity-sensitive distance function also presupposes access to a rich, discriminative representation for the objects at hand. In this dissertation we present contributions towards both ends. In the first part of the thesis, assuming good representations for the data, we present a formulation for metric learning that makes a more direct attempt to optimize for the k-NN accuracy as compared to prior work. We also present extensions of this formulation to metric learning for kNN regression, asymmetric similarity learning and discriminative learning of Hamming distance. In the second part, we consider a situation where we are on a limited computational budget i.e. optimizing over a space of possible metrics would be infeasible, but access to a label aware distance metric is still desirable. We present a simple, and computationally inexpensive approach for estimating a well motivated metric that relies only on gradient estimates, discussing theoretical and experimental results. In the final part, we address representational issues, considering group equivariant convolutional neural networks (GCNNs). Equivariance to symmetry transformations is explicitly encoded in GCNNs; a classical CNN being the simplest example. In particular, we present a SO(3)-equivariant neural network architecture for spherical data, that operates entirely in Fourier space, while also providing a formalism for the design of fully Fourier neural networks that are equivariant to the action of any continuous compact group.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑