arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2307.12754stat.MEcs.AIcs.LGmath.STstat.TH
收藏

通过正则化实现回归中的非参数线性特征学习

Nonparametric Linear Feature Learning in Regression Through Regularisation

Inria(法国国家信息与自动化研究所);PSL Research University(巴黎文理研究大学)

Bertille Follain, Francis Bach

AI总结:

本文提出RegFeaL方法,通过联合线性特征学习与非参数函数估计,利用Hermite多项式和交替最小化在高维多指标模型中实现高效特征选择,并证明其风险收敛性。

摘要:

表示学习在自动化特征选择中起着至关重要的作用,尤其是在高维数据背景下,非参数方法往往面临困难。在本研究中,我们关注监督学习场景,其中相关信息位于数据的低维线性子空间中,即多指标模型。如果该子空间已知,将极大提升预测、计算和可解释性。为应对这一挑战,我们提出了一种联合线性特征学习和非参数函数估计的新方法,旨在更有效地利用隐藏特征进行学习。我们的方法采用经验风险最小化,并对函数导数施加惩罚,从而保证通用性。利用Hermite多项式的正交性和旋转不变性,我们引入了名为RegFeaL的估计器。通过交替最小化,我们迭代旋转数据以改善与主导方向的对齐。我们证明,在最小假设下,我们的方法的期望风险以高概率收敛到最小风险,并具有显式收敛速率。此外,我们提供了实证结果,展示了RegFeaL在多种实验中的性能。

英文摘要:

Representation learning plays a crucial role in automated feature selection, particularly in the context of high-dimensional data, where non-parametric methods often struggle. In this study, we focus on supervised learning scenarios where the pertinent information resides within a lower-dimensional linear subspace of the data, namely the multi-index model. If this subspace were known, it would greatly enhance prediction, computation, and interpretation. To address this challenge, we propose a novel method for joint linear feature learning and non-parametric function estimation, aimed at more effectively leveraging hidden features for learning. Our approach employs empirical risk minimisation, augmented with a penalty on function derivatives, ensuring versatility. Leveraging the orthogonality and rotation invariance properties of Hermite polynomials, we introduce our estimator, named RegFeaL. By using alternative minimisation, we iteratively rotate the data to improve alignment with leading directions. We establish that the expected risk of our method converges in high-probability to the minimal risk under minimal assumptions and with explicit rates. Additionally, we provide empirical results demonstrating the performance of RegFeaL in various experiments.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑