核特征选择中的非凸性驯服--Laplace核的有利性质
Taming Nonconvexity in Kernel Feature Selection -- Favorable Properties of the Laplace Kernel
摘要：
基于核的特征选择是非参数统计中的一个重要工具。尽管基于核的特征选择有许多实际应用，但很少有统计理论支持这种方法。核心挑战是用于定义基于核的特征选择的最优化问题的目标函数是非凸的。文献只研究了\emph{global optima}的统计特性，这是一个不匹配的，因为基于梯度的算法可用于非凸优化只能保证收敛到局部最小值。研究与基于内核的方法相关的完整景观，我们表明，使用拉普拉斯内核（和其他$\ell_1$内核）的特征选择目标来与其他内核，包括无处不在的高斯内核（或其他$\ell_2$内核）不具备的统计保证。基于目标函数的梯度的尖锐表征，我们表明，$\ell_1$内核消除不利的固定点时，出现使用$\ell_2$内核。有了这个洞察力，我们建立了统计保证$\ell_1$基于内核的功能选择，不需要达到全球最小值。特别是，我们建立了模型选择的一致性$\ell_1$ -基于内核的功能选择恢复主效应和层次的相互作用，在非参数设置与$n \sim \log p$样本。
英文摘要：
Kernel-based feature selection is an important tool in nonparametric statistics. Despite many practical applications of kernel-based feature selection, there is little statistical theory available to support the method. A core challenge is the objective function of the optimization problems used to define kernel-based feature selection are nonconvex. The literature has only studied the statistical properties of the \emph{global optima}, which is a mismatch, given that the gradient-based algorithms available for nonconvex optimization are only able to guarantee convergence to local minima. Studying the full landscape associated with kernel-based methods, we show that feature selection objectives using the Laplace kernel (and other $\ell_1$ kernels) come with statistical guarantees that other kernels, including the ubiquitous Gaussian kernel (or other $\ell_2$ kernels) do not possess. Based on a sharp characterization of the gradient of the objective function, we show that $\ell_1$ kernels eliminate unfavorable stationary points that appear when using an $\ell_2$ kernel. Armed with this insight, we establish statistical guarantees for $\ell_1$ kernel-based feature selection which do not require reaching the global minima. In particular, we establish model-selection consistency of $\ell_1$-kernel-based feature selection in recovering main effects and hierarchical interactions in the nonparametric setting with $n \sim \log p$ samples.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
