arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1206.6397cs.LGstat.ML
收藏

通信启发线性判别分析

Communications Inspired Linear Discriminant Analysis

Duke University(杜克大学);PA Consulting Group(PA咨询集团);University College London(伦敦大学学院)

Minhua Chen, William Carson, Miguel Rodrigues, Robert Calderbank, Lawrence Carin

摘要:

本文从信息论的角度研究了有监督的线性降维问题。通过最大化投影信号和类别标签之间的互信息(基于香农熵度量)来设计线性投影矩阵。通过利用最近关于互信息的梯度的理论结果,可以直接使用梯度下降来解决上述优化问题,而不需要简化目标函数。对该方法与两种密切相关的方法(线性判别分析和信息判别分析)进行了理论分析和实证比较,并与用Renyi熵定义互信息的方法进行了比较(在这种情况下,在特定的参数设置下,可以简单地计算梯度).相对于这些替代方法,所提出的方法在真实数据集上取得了令人鼓舞的结果。

英文摘要:

We study the problem of supervised linear dimensionality reduction, taking an information-theoretic viewpoint. The linear projection matrix is designed by maximizing the mutual information between the projected signal and the class label (based on a Shannon entropy measure). By harnessing a recent theoretical result on the gradient of mutual information, the above optimization problem can be solved directly using gradient descent, without requiring simplification of the objective function. Theoretical analysis and empirical comparison are made between the proposed method and two closely related methods (Linear Discriminant Analysis and Information Discriminant Analysis), and comparisons are also made with a method in which Renyi entropy is used to define the mutual information (in this case the gradient may be computed simply, under a special parameter setting). Relative to these alternative approaches, the proposed method achieves promising results on real datasets.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑