二次互信息导数的直接估计及其在监督降维中的应用
Direct Estimation of the Derivative of Quadratic Mutual Information with Application in Supervised Dimension Reduction
摘要：
监督降维的一个典型目标是找到输入空间的低维子空间，使得投影的输入变量保留关于输出变量的最大信息。依赖最大化方法通过最大化投影输入变量和输出变量之间的统计依赖来解决监督降维问题。一个著名的统计相关性度量是基于Kullback-Leibler（KL）散度的互信息（MI）。然而，已知库尔贝克-莱布勒散度对异常值敏感。另一方面，二次MI（QMI）是基于$L_2$距离的MI的变体，其比库尔贝克-莱布勒散度对离群值更鲁棒，并且最近已经提出了一种从数据估计QMI的计算有效的方法，称为最小二乘QMI（LSQMI）。基于这些原因，发展一种基于LSQMI的有监督的降维方法似乎是有希望的。然而，在有监督的降维中，子空间搜索需要的不是QMI本身，而是QMI的导数，并且精确的QMI估计的导数不一定是QMI导数的好估计。在本文中，我们提出了直接估计QMI的导数，而不估计QMI本身。我们表明，直接估计的QMI的衍生物是更准确的比衍生物的估计QMI。最后，我们开发了一个有监督的降维算法，有效地使用所提出的导数估计，并通过实验证明，所提出的方法是更强大的离群值比现有的方法。
英文摘要：
A typical goal of supervised dimension reduction is to find a low-dimensional subspace of the input space such that the projected input variables preserve maximal information about the output variables. The dependence maximization approach solves the supervised dimension reduction problem through maximizing a statistical dependence between projected input variables and output variables. A well-known statistical dependence measure is mutual information (MI) which is based on the Kullback-Leibler (KL) divergence. However, it is known that the KL divergence is sensitive to outliers. On the other hand, quadratic MI (QMI) is a variant of MI based on the $L_2$ distance which is more robust against outliers than the KL divergence, and a computationally efficient method to estimate QMI from data, called least-squares QMI (LSQMI), has been proposed recently. For these reasons, developing a supervised dimension reduction method based on LSQMI seems promising. However, not QMI itself, but the derivative of QMI is needed for subspace search in supervised dimension reduction, and the derivative of an accurate QMI estimator is not necessarily a good estimator of the derivative of QMI. In this paper, we propose to directly estimate the derivative of QMI without estimating QMI itself. We show that the direct estimation of the derivative of QMI is more accurate than the derivative of the estimated QMI. Finally, we develop a supervised dimension reduction algorithm which efficiently uses the proposed derivative estimator, and demonstrate through experiments that the proposed method is more robust against outliers than existing methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
