线性回归中加权数据选择的精确风险比

中文摘要 AI 辅助

Hanneke、Moran、Shlimovich与Yehudayoff（COLT 2025）提出了如下开放问题：选择器观察有限数据集$D \subseteq \mathbb{R}^d \times \mathbb{R}$，选取至多$n$个样本并赋予非负权重，将加权最小二乘目标函数交由最小范数经验风险最小化（ERM）求解。记$F_w(d,n)$为返回预测器在全数据集$D$上的损失与最优损失的最坏情况比值，他们证明：当$n<d$时$F_w(d,n)=\infty$，当$n=d$时$F_w(d,d)=d+1$，当$n \ge 2d$时$F_w(d,n)=1$，并询问在开放区间$d<n<2d$内该值的大小。我们在若干情形下确定了该值：对任意$d$，证明$F_w(d,2d-1)=1+1/d$，验证了原始注记中未加证明的结论；进一步证明$F_w(3,4)=5/3$、$F_w(4,5)=2$，这是端点公式未覆盖的两个最小单元；对任意中间预算$n=d+k$，证明下界$F_w(d,d+k) \ge 1+\Gamma_{d,k}$，其中$\Gamma_{d,k}$是平衡划分上的显式调和量，且表明该界是白化梯度系统具有正交回路块结构的数据集类上的精确极小极大值。所有三个精确值均符合$1+\Gamma_{d,k}$，我们猜想在整个开放区间内等式成立。上界证明基于共同的几何核心：损失梯度正张成构型的刚性定理、$\mathbb{R}^3$与$\mathbb{R}^4$中小正基的分类与结构归约，以及将符号锥几何转化为五点选择的无维极值基论证。我们还给出明确反例表明若干更短路径不成立，并为所有已证明情形构造了多项式时间选择算法。

英文摘要

How much data must a fixed learner retain? Hanneke, Moran, Shlimovich and Yehudayoff (COLT 2025) posed this question for linear regression with the minimum-norm empirical risk minimizer. A selector sees a finite dataset $D\subseteq R^d\times R$, keeps at most $n$ examples with nonnegative weights, and $F_w(d,n)$ is the worst-case ratio between the full-data loss of the trained predictor and the optimal loss. The value is $\infty$ for $n<d$, $d+1$ at $n=d$ and $1$ for $n\ge2d$, and the regime $d<n<2d$ was left open. We settle several cases. For every $d$ we prove $F_w(d,2d-1)=1+1/d$, which confirms a claim stated without proof in the original note. We also prove $F_w(3,4)=5/3$, $F_w(4,5)=2$ and $F_w(4,6)=3/2$, the three smallest cells not covered by that formula. For every intermediate budget $n=d+k$ we prove the lower bound $F_w(d,d+k)\ge1+Γ_{d,k}$, where $Γ_{d,k}$ is an explicit harmonic quantity over balanced partitions of $d$. This bound is the exact minimax value on the class of datasets whose whitened systems split into orthogonal circuit blocks. All proved values equal $1+Γ_{d,k}$, and we conjecture that this holds throughout the open regime. Our upper bounds combine a rigidity theorem for positive spanning configurations of loss gradients with normal forms of the small positive bases in $R^3$ and $R^4$. These forms are special cases of the classification of Cornaz, Kerleau and Royer; we also control all gradients outside the basis. A dimension-free extremal-basis argument converts sign-cone geometry into selections of $d+1$ points. Explicit counterexamples rule out several shorter routes. Every upper bound is constructive, with selection procedures polynomial in the number of points for fixed dimension. Every numerical claim about a specific instance is an exact rational or algebraic identity, recomputed in exact arithmetic in the supplementary material.

发表机构

• University of New South Wales（新南威尔士大学）

机构由 AI 辅助整理，请以论文原文为准。

补充信息

备注

48 pages