arXiv 2411.096862024-11stat.MLcs.LG

非线性单变量模型的条件回归

Conditional regression for the Nonlinear Single-Variable Model

Department of Mathematics Johns Hopkins University（数学系约翰霍普金斯大学）

Yantao Wu, Mauro Maggioni

备注：: 75 pages, 13 figures

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出了一种非参数条件回归估计器，用于非线性单变量模型，通过投影到曲线参数来避免维度诅咒，达到最优回归速率。

摘要：

对函数$F$在$\mathbb{R}^d$上的回归，而不受统计和计算维度诅咒，需要特殊的统计模型，例如对数据分布施加几何假设（例如其支撑是低维的），或对$F$施加强光滑性假设，或对$F$施加特殊结构。其中，复合模型$F=f\circ g$，其中$g$映射到$\mathbb{R}^r$，且$r\ll d$，包括经典的单指数模型和多指数模型，以及神经网络。虽然当$g$是线性时情况已很明确，但当$g$是非线性时了解较少，特别是在哪些$g$的情况下可以避免估计$F$的维度诅咒，或同时避免$F$和$g$的维度诅咒。本文考虑模型$F(X):=f(Π_γX)$，其中$Π_γ:\mathbb{R}^d\to[0,\textrm{len}_γ]$是最近点投影到曲线$γ:[0, \textrm{len}_γ]\to\mathbb{R}^d$的参数上，而$f:[0,\textrm{len}_γ]\to \mathbb{R}^1$。输入数据$X$不是低维的：它可以离$γ$的距离达到条件$Π_γ(X)$是良好定义所允许的程度。分布$X$、曲线$γ$和函数$f$都是未知的。该模型是单指数模型的自然非线性推广，对应于$γ$是直线的情况。我们提出了一种非参数估计器，基于条件回归，在适当假设下，最强的假设是$f$是粗略单调的，可以达到非参数回归的一维最优min-max速率，直到观测中的噪声水平，并且可以在时间$\mathcal{O}(d^2 n\log n)$内构建。学习界中的所有常数、所需样本的最小数量以及计算复杂度都是至多低阶多项式形式的$d$。

英文摘要：

Regressing a function $F$ on $\mathbb{R}^d$ without incurring the statistical and computational curse of dimensionality requires exploitable structure. Compositional models $F=f\circ g$ in which $g$ has a low-dimensional range include classical single- and multi-index models as well as certain neural networks; while the case of linear $g$ is well understood, substantially less is known for nonlinear $g$. We study the model $F(X)=f(Π_γX)$, where $Π_γ$ is the closest-point coordinate associated with an unknown regular curve $γ$, and $f$ is an unknown one-dimensional link function. The predictor $X$ need not be intrinsically low-dimensional and may have full-dimensional variation throughout a tubular neighborhood of the curve. We construct a nonparametric estimator based on response slicing, local principal component analysis, data-adaptive slice assignment, and one-dimensional local polynomial regression. Under coarse monotonicity of $f$ and sufficient variation normal to the curve relative to the observational noise and the coarse-monotonicity scale, the estimator attains, up to logarithmic factors, the minimax-optimal one-dimensional mean squared rate down to an explicit geometry- and noise-dependent saturation level. When the normal-variation condition is removed, we prove a complementary guarantee for the wide-slice regime. The estimator can be constructed in time $\mathcal{O}(d^2n\log n)$, and the constants and sample-size thresholds in our bounds depend at most polynomially on the ambient dimension $d$.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

AI总结：

摘要：

英文摘要：

相关论文