School of Automation, Northwestern Polytechnical University(西北工业大学自动化学院);Information Engineering College of Yangzhou University(扬州大学信息工程学院);State Key Lab of Intelligent Technologies and Systems, Beijing National Research Center for Information Science and Technology(智能技术与系统国家重点实验室,北京信息科学与技术国家研究中心);Department of Automation, Tsinghua University(清华大学自动化系)
Kun Song, Junwei Han, Gong Cheng, Jiwen Lu, Feiping Nie
在本文中,我们揭示了度量学习将遭受严重的不可分割的问题,如果没有信息样本挖掘。由于不可分样本中经常会混入硬样本,现有的信息样本挖掘策略在处理不可分样本时可能会带来一些副作用,如目标函数不稳定等。针对这一问题,提出了一种新的距离度量学习算法,即自适应邻域度量学习(ANML)。在ANML中,我们设计了两个阈值来自适应地识别训练过程中不可分的相似和不相似样本,从而在同一过程中实现不可分样本去除和度量参数学习。由于所提出的ANML的非连续性,我们开发了一个巧妙的功能,命名为\emph{log-exp mean function}构建一个连续的配方来代替它,它可以有效地解决了梯度下降方法。与Triplet loss类似,ANML可用于学习线性嵌入和深度嵌入。通过分析所提出的方法,我们发现它有一些有趣的性质。例如,当ANML用于学习线性嵌入时,当前著名的度量学习算法如大间隔最近邻(LMNN)和邻域分量分析(NCA)通过设置不同的参数值而成为所提出的ANML的特例。当它被用来学习深度特征时,最先进的深度度量学习算法如Triplet loss、Lifted structure loss和Multi-similarity loss成为ANML的特例。此外,在我们的方法中提出的\emph{log-exp mean function}提供了一个新的视角来回顾深度度量学习方法,如Prox-NCA和N对损失。最后通过实验验证了该方法的有效性。
英文摘要:
In this paper, we reveal that metric learning would suffer from serious inseparable problem if without informative sample mining. Since the inseparable samples are often mixed with hard samples, current informative sample mining strategies used to deal with inseparable problem may bring up some side-effects, such as instability of objective function, etc. To alleviate this problem, we propose a novel distance metric learning algorithm, named adaptive neighborhood metric learning (ANML). In ANML, we design two thresholds to adaptively identify the inseparable similar and dissimilar samples in the training procedure, thus inseparable sample removing and metric parameter learning are implemented in the same procedure. Due to the non-continuity of the proposed ANML, we develop an ingenious function, named \emph{log-exp mean function} to construct a continuous formulation to surrogate it, which can be efficiently solved by the gradient descent method. Similar to Triplet loss, ANML can be used to learn both the linear and deep embeddings. By analyzing the proposed method, we find it has some interesting properties. For example, when ANML is used to learn the linear embedding, current famous metric learning algorithms such as the large margin nearest neighbor (LMNN) and neighbourhood components analysis (NCA) are the special cases of the proposed ANML by setting the parameters different values. When it is used to learn deep features, the state-of-the-art deep metric learning algorithms such as Triplet loss, Lifted structure loss, and Multi-similarity loss become the special cases of ANML. Furthermore, the \emph{log-exp mean function} proposed in our method gives a new perspective to review the deep metric learning methods such as Prox-NCA and N-pairs loss. At last, promising experimental results demonstrate the effectiveness of the proposed method.
Neighborhood-Adaptive Generalized Linear Graph Embedding with Latent Pattern Mining
School of Computer and Information, Anhui Normal University(安徽师范大学计算机与信息学院);Anhui Provincial Key Laboratory of Industrial Intelligent Data Security(安徽省工业智能数据安全重点实验室);Institute of Automation, State Key Laboratory of Multimodal Artificial Intelligence Systems, Chinese Academy of Sciences(自动化研究所,多模态人工智能系统国家重点实验室,中国科学院);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院)
ParBalans: Parallel Multi-Armed Bandits-based Adaptive Large Neighborhood Search
Department of Mechanical & Industrial Engineering, Northeastern University(东北大学机械与工业工程系);AI Center of Excellence, Fidelity Investments(fidelity investments 精英研究中心);Department of Computer Science, University of Southern California(南加州大学计算机科学系);Department of Computer Science, Brown University(布朗大学计算机科学系)
Balans: Multi-Armed Bandits-based Adaptive Large Neighborhood Search for Mixed-Integer Programming Problem
Department of Computer Science, University of Southern California(南加州大学计算机科学系);AI Center of Excellence, Fidelity Investments(富达投资人工智能卓越中心);Department of Computer Science, Brown University(布朗大学计算机科学系)
Ferdowsi University of Mashhad(马什哈德费尔多西大学);Center of Excellence in Soft Computing and Intelligent Information Processing, Ferdowsi University of Mashhad(马什哈德费尔多西大学软计算与智能信息处理卓越中心)
New Adaptive Mechanism for Large Neighborhood Search using Dual Actor-Critic
School of Intelligent Science and Technology, Nanjing University of Science and Technology(智能科学与技术学院,南京理工大学);Department of Mechanical Engineering, North China Electric Power University, Baoding 071003, China(机械工程系,华北电力大学,保定071003,中国);EM Normandie Business School, Métis Lab, 92110, Clichy, France(EM诺曼底商学院,Métis实验室,92110,法国);Université Paris-Saclay, CentraleSupélec, Laboratoire Génie Industriel, 91190, Gif-sur-Yvette, France(巴黎萨克雷大学,CentraleSupélec,工业工程实验室,91190,法国)
IncA-DES: An incremental and adaptive dynamic ensemble selection approach using online K-d tree neighborhood search for data streams with concept drift
LIVIA, École de Technologie Supérieure, University of Qu\'ebec, Montreal, Qu\'ebec, Canada;Department of Informatics, Universidade Federal do Paran\' a (UFPR), Curitiba (PR), Brazil
HyP-ASO: A Hybrid Policy-based Adaptive Search Optimization Framework for Large-Scale Integer Linear Programs
Ning Xu School of Mathematics and Statistics Nanjing University of Science and Technology(南京理工大学数学与统计学学院);Junkai Zhang Institute of Automation Chinese Academy of Sciences(中国科学院自动化研究所);Huigen Ye Department of Computer Science and Technology Tsinghua University(清华大学计算机科学与技术系);Yang Wu Institute of Automation Chinese Academy of Sciences(中国科学院自动化研究所);Hua Xu Department of Computer Science and Technology Tsinghua University(清华大学计算机科学与技术系);Huiling Xu School of Mathematics and Statistics Nanjing University of Science and Technology(南京理工大学数学与统计学学院);Yifan Zhang Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所)