arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2102.05836cs.LG
收藏

用于分类和聚类的在线确定性退火

Online Deterministic Annealing for Classification and Clustering

University of Maryland(马里兰大学)

Christos Mavridis, John Baras

AI总结:

提出一种在线无梯度原型学习算法,通过退火过程避免局部最优、增强鲁棒性并渐进增加模型复杂度,实现分类与聚类,且需极少超参数调优,并揭示Bregman散度的核心作用。

摘要:

几乎每一个迭代式机器学习算法都固有地存在超参数调优问题,其中包括三个主要设计参数:(a)模型的复杂度,例如神经网络中神经元的数量;(b)初始条件,它严重影响算法的行为;(c)用于量化其性能的相异性度量。我们提出了一种基于原型的在线学习算法,该算法可被视为一种用于分类和聚类的渐进式增长的竞争学习神经网络架构。所提方法的学习规则被表述为一种在线无梯度随机逼近算法,该算法求解一系列适当定义的优化问题,模拟退火过程。算法的退火特性有助于避免较差的局部最小值,对初始条件具有鲁棒性,并通过直观的分叉现象提供了一种逐步增加学习模型复杂度的方法。所提方法具有可解释性,需要最少的超参数调优,并允许在线控制性能与复杂度之间的权衡。最后,我们证明了Bregman散度作为一族相异性度量自然出现,在算法的性能和计算复杂度方面都发挥着核心作用。

英文摘要:

Inherent in virtually every iterative machine learning algorithm is the problem of hyper-parameter tuning, which includes three major design parameters: (a) the complexity of the model, e.g., the number of neurons in a neural network, (b) the initial conditions, which heavily affect the behavior of the algorithm, and (c) the dissimilarity measure used to quantify its performance. We introduce an online prototype-based learning algorithm that can be viewed as a progressively growing competitive-learning neural network architecture for classification and clustering. The learning rule of the proposed approach is formulated as an online gradient-free stochastic approximation algorithm that solves a sequence of appropriately defined optimization problems, simulating an annealing process. The annealing nature of the algorithm contributes to avoiding poor local minima, offers robustness with respect to the initial conditions, and provides a means to progressively increase the complexity of the learning model, through an intuitive bifurcation phenomenon. The proposed approach is interpretable, requires minimal hyper-parameter tuning, and allows online control over the performance-complexity trade-off. Finally, we show that Bregman divergences appear naturally as a family of dissimilarity measures that play a central role in both the performance and the computational complexity of the learning algorithm.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑