用于增强二分类性能的协方差矩阵与Hessian矩阵协同特征分析
Synergistic eigenanalysis of covariance and Hessian matrices for enhanced binary classification
AI总结:
本研究提出协方差与Hessian矩阵协同特征分析的二分类新方法,通过形式化证明其兼顾LDA的分离性与紧凑性准则,在多数据集上性能优于PCA、Hessian方法及LDA,还可助力理解DNN决策。
摘要:
现有文献中针对分类问题的协方差矩阵与Hessian矩阵研究均是独立开展的,但将二者结合有望提升协同效用,改善分类性能。本文提出一种新方法,将训练集上计算得到的协方差矩阵的特征分析,与深度学习模型上计算得到的Hessian矩阵的特征分析相结合,在二分类任务中实现最优类别可分性。\n我们通过形式化证明验证了该方法的有效性:它能够最大化类间均值距离(即“分离性”概念)、最小化类内方差(即“紧凑性”概念),这两项正是线性判别分析(LDA)的两大准则,尤其在类均值周围各向同性、主导特征值显著等理想数据条件下效果更突出。通过将数据投影到由两类矩阵最相关特征方向构成的联合空间中,我们实现了符合上述LDA准则的最优类别可分性。\n在神经科学数据集与医疗健康数据集上的实验验证一致支持了我们的理论框架,结果表明该方法性能优于现有成熟方法。与分别侧重单一准则的PCA和Hessian方法不同,本方法同时兼顾分离性与紧凑性两大准则,这种全面的方式能够捕捉复杂的模式与关联,提升分类性能。此外,通过同时利用两大LDA准则,本方法借助高维特征空间的优势性能超越了LDA本身,这符合Cover定理——高维空间更易实现线性可分。另外,该方法还为理解深度神经网络(DNN)的复杂决策过程提供了新思路,可在二维空间中呈现其决策逻辑。
英文摘要:
Covariance and Hessian matrices have been analyzed separately in the literature for classification problems. However, integrating these matrices has the potential to enhance their combined power in improving classification performance. We present a novel approach that combines the eigenanalysis of a covariance matrix evaluated on a training set with a Hessian matrix evaluated on a deep learning model to achieve optimal class separability in binary classification tasks. Our approach is substantiated by formal proofs that establish its capability to maximize between-class mean distance (the concept of \textit{separation}) and minimize within-class variances (the concept of \textit{compactness}), which together define the two linear discriminant analysis (LDA) criteria, particularly under ideal data conditions such as isotropy around class means and dominant leading eigenvalues. By projecting data into the combined space of the most relevant eigendirections from both matrices, we achieve optimal class separability as per these LDA criteria. Empirical validation across neural and health datasets consistently supports our theoretical framework and demonstrates that our method outperforms established methods. Our method stands out by addressing both separation and compactness criteria, unlike PCA and the Hessian method, which predominantly emphasize one criterion each. This comprehensive approach captures intricate patterns and relationships, enhancing classification performance. Furthermore, through the utilization of both LDA criteria, our method outperforms LDA itself by leveraging higher-dimensional feature spaces, in accordance with Cover's theorem, which favors linear separability in higher dimensions. Additionally, our approach sheds light on complex DNN decision-making, rendering them comprehensible within a 2D space.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。