arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2001.07026stat.MLcs.CVcs.LG

利用张量核函数减少深度聚类中的目标函数不匹配

Leveraging tensor kernels to reduce objective function mismatch in deep clustering

NORCE Norwegian Research Centre(挪威研究中心(NORCE));UiT The Arctic University of Norway(挪威北极大学);University of Copenhagen(哥本哈根大学);Norwegian Computing Center(挪威计算中心)

Daniel J. Trosten, Sigurd Løkse, Robert Jenssen, Michael Kampffmeyer

AI总结:

针对深度聚类中基于自编码器方法导致的目标函数不匹配问题,提出利用张量核函数的无监督伴随目标(UCOs),有效降低了目标间的不匹配并提升了聚类性能。

摘要:

目标函数不匹配(Objective Function Mismatch, OFM)是指一个目标的优化对另一个目标的优化产生负面影响的现象。在本研究中,我们探讨了深度聚类中的OFM问题,并发现基于自编码器的流行深度聚类方法不仅会导致聚类性能下降,还会在重构目标和聚类目标之间引发显著的OFM。为了在保持辅助目标结构保留特性的同时减少这种不匹配,我们提出了一组用于深度聚类的新辅助目标,称为无监督伴随目标(Unsupervised Companion Objectives, UCOs)。UCOs依赖核函数在网络中间表示上构建聚类目标。通常,除了特征维度外,中间表示还可能包含其他维度,例如空间或时间维度。因此我们认为,将这些表示向量化并应用向量核的朴素方法是次优的,因为它忽略了其他维度中包含的信息。为解决这一缺陷,我们为UCOs配备了适用于任意阶张量的结构利用张量核。因此,UCOs能够适应广泛的网络架构类别。我们还提出了一种新颖的、基于回归的OFM度量方法,使我们能够准确量化训练过程中观测到的OFM量。实验表明,与类似的基于自编码器的模型相比,UCOs与主聚类目标之间的OFM更低。此外,我们证明与基于自编码器的方法相比,UCOs提升了模型的聚类性能。我们的实验代码可在 https://github.com/danieltrosten/tk-uco 获取。

英文摘要:

Objective Function Mismatch (OFM) occurs when the optimization of one objective has a negative impact on the optimization of another objective. In this work we study OFM in deep clustering, and find that the popular autoencoder-based approach to deep clustering can lead to both reduced clustering performance, and a significant amount of OFM between the reconstruction and clustering objectives. To reduce the mismatch, while maintaining the structure-preserving property of an auxiliary objective, we propose a set of new auxiliary objectives for deep clustering, referred to as the Unsupervised Companion Objectives (UCOs). The UCOs rely on a kernel function to formulate a clustering objective on intermediate representations in the network. Generally, intermediate representations can include other dimensions, for instance spatial or temporal, in addition to the feature dimension. We therefore argue that the naïve approach of vectorizing and applying a vector kernel is suboptimal for such representations, as it ignores the information contained in the other dimensions. To address this drawback, we equip the UCOs with structure-exploiting tensor kernels, designed for tensors of arbitrary rank. The UCOs can thus be adapted to a broad class of network architectures. We also propose a novel, regression-based measure of OFM, allowing us to accurately quantify the amount of OFM observed during training. Our experiments show that the OFM between the UCOs and the main clustering objective is lower, compared to a similar autoencoder-based model. Further, we illustrate that the UCOs improve the clustering performance of the model, in contrast to the autoencoder-based approach. The code for our experiments is available at https://github.com/danieltrosten/tk-uco.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑