基于级集聚合的传输学习乘法Oracle不等式
Multiplicative Oracle Inequalities for Transductive Learning via Level-Set Aggregation
AI总结：
本文研究了传输学习中基于级集聚合的乘法Oracle不等式，针对多种任务如分类、回归和密度估计，提出了改进的不等式形式，通过级集聚合方法提升了预测性能。
AI中文摘要：
我们重新审视了传输学习，其中预测是在所有协变量已知的情况下进行的。在留一法（LOO）设置中，预测使用剩余样本点的标签并评估平均误差。本文研究了各种任务中对无偏传输LOO预测的乘法Oracle不等式，包括0-1损失分类、平方损失回归、密度估计和逻辑回归。我们引入了基于近似经验风险最小化（ERM）级集的中位数级集聚合（MLSA）方法。我们证明了一个通用的乘法Oracle不等式，形式为LOO_S(MLSA) ≤ C(1/n min_{h∈H} L_S(h) + log|H|/n)，其中H是假设函数类。该不等式适用于满足局部级集增长条件的假设类以及满足轻微单调性假设的损失函数。对于VC类的分类在0-1损失下，log|H|因子可改进为d log n，其中d是VC维数，恢复Long (1998)的结果至log n因子。对于具有受限协变量和参数的逻辑回归，log|H|因子可改进为d log n，至问题相关的因子。
英文摘要：
We revisit transductive learning where predictions are made with the set of all covariates known in advance. In the leave-one-out (LOO) setting, the prediction is made with labels of the remaining sample points and evaluated by the average error. In particular, we study multiplicative oracle inequalities for agnostic transductive LOO prediction for a variety of tasks, including classification with 0-1 loss, squared loss regression, density estimation, and logistic regression. Specifically, we introduce \emph{Median of Level-Set Aggregation} (MLSA), an aggregation procedure built on near-ERM level sets (i.e., empirical-risk level sets around the ERM). We prove a general multiplicative oracle inequality for the LOO error of the form \[ LOO_S(MLSA) \;\le\; C \left( \frac{1}{n} \min_{h\in H} L_S(h) \;+\; \frac{\log |H|}{n}\right), \qquad C>1, \] where $H$ is the hypothesis/function class. This inequality holds for hypothesis classes under a local level-set growth condition together with losses satisfying a mild monotonicity assumption. For classification with VC classes under the $0$--$1$ loss, the $\log |H|$ factor can be improved to be $d\log n$, where $d$ is the VC dimension, recovering Long (1998) up to a $\log n$ factor. For logistic regression with bounded covariates and parameters, the $\log |H|$ factor can be improved to be $d\log n$ up to problem-dependent factors, where $d$ is the ambient dimension.
