在过去的几十年里,距离度量学习在解决机器学习和模式识别问题方面吸引了很多兴趣。在这项工作中,我们提出了一个简单的方法,从统计物理学的概念,学习最佳的距离度量给定的问题。我们制定的任务作为一个典型的统计物理问题:模式之间的距离代表的组成部分的物理系统和目标函数对应的能量。然后,我们将问题表示为复杂系统的自由能的最小化,这相当于距离度量学习。就像物理学中的许多问题一样,我们提出了一种基于Metropolis Monte Carlo的方法来找到最佳距离度量。这提供了一种自然的方式来学习距离度量,其中学习过程可以直观地看作是拉伸和旋转度量空间,直到满足某些启发式。我们提出的方法可以处理各种各样的约束,包括那些虚假的局部极小值。该方法与来自邻域分量分析(NCA)的随机最近邻的效果令人惊讶地好。人工和真实世界的数据集上的实验结果显示出明显的优越性,在一些国家的最先进的距离度量学习方法的最近邻分类。
英文摘要:
Distance metric learning has attracted a lot of interest for solving machine learning and pattern recognition problems over the last decades. In this work we present a simple approach based on concepts from statistical physics to learn optimal distance metric for a given problem. We formulate the task as a typical statistical physics problem: distances between patterns represent constituents of a physical system and the objective function corresponds to energy. Then we express the problem as a minimization of the free energy of a complex system, which is equivalent to distance metric learning. Much like for many problems in physics, we propose an approach based on Metropolis Monte Carlo to find the best distance metric. This provides a natural way to learn the distance metric, where the learning process can be intuitively seen as stretching and rotating the metric space until some heuristic is satisfied. Our proposed method can handle a wide variety of constraints including those with spurious local minima. The approach works surprisingly well with stochastic nearest neighbors from neighborhood component analysis (NCA). Experimental results on artificial and real-world data sets reveal a clear superiority over a number of state-of-the-art distance metric learning methods for nearest neighbors classification.
Gap Minimization for Knowledge Sharing and Transfer
University of Western Ontario(西安大略大学);Massachusetts Institute of Technology(麻省理工学院);Université Laval(拉瓦尔大学);Beihang University(北京航空航天大学);McGill University(麦吉尔大学);University of Pennsylvania(宾夕法尼亚大学)
Graph Neural Distance Metric Learning with Graph-Bert
Florida State University(佛罗里达州立大学);IFM Lab(IFM实验室)
图距离度量学习是许多图学习问题的基础,例如,图聚类、图分类和图匹配。现有的关于图距离度量(或图核)学习的研究工作未能保持这些度量的基本性质,例如,非负性、不可约恒等式、对称性和三角不等式。在本文中,我们将介绍一种新的图神经网络的距离度量学习方法,即GB-DISTANCE(GRAPH-BERT based Neural Distance)。基于注意力机制,GB-DISTANCE可以基于预先训练的GRAPH-BERT模型有效地学习图实例表示。与现有的监督/无监督度量不同,GB-DISTANCE可以以半监督的方式有效地学习。此外,GB-DISTANCE还可以保持上述距离度量的基本属性。在多个基准图数据集上进行了大量实验,结果表明GB-DISTANCE算法的性能优于现有的基线方法,尤其是最近提出的基于图度量的图神经网络模型,但在计算图距离方面存在明显差距。
Learning Domain Invariant Representations by Joint Wasserstein Distance Minimization
Technische Universität Berlin(柏林工业大学);Berlin Institute for the Foundations of Learning and Data – BIFOLD(柏林学习与数据基础研究所(BIFOLD));Tokyo Institute of Technology(东京工业大学);Fujitsu Laboratories Ltd.(富士通实验室有限公司);RIKEN AIP(理化学研究所人工智能研究中心);Max Planck Institute for Informatics(马克斯·普朗克信息学研究所);Korea University(高丽大学);Google Deepmind(谷歌DeepMind);Freie Universität Berlin(柏林自由大学)
The Best Time for an Update: Risk-Sensitive Minimization of Age-Based Metrics
Communications Engineering Lab, Technical University of Darmstadt(达姆施塔特工业大学通信工程实验室);Probability and Statistics Group, Mathematics Departement, Technical University of Darmstadt(达姆施塔特工业大学数学系概率与统计组)