对抗性响应下的不可知平滑在线回归
Agnostic Smoothed Online Regression with Adversarial Responses
- University of Michigan（密歇根大学）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
针对对抗性响应的平滑在线回归，提出Hedge-Cover算法，实现次线性遗憾并匹配下界，解决开放问题，并量化噪声影响。
AI中文摘要：
我们研究了具有有界对抗性响应的平滑在线预测问题。这一被广泛研究的框架通过平滑参数 $\textsf{C}_{\textsf{cov}}$ 将独立同分布采样与对抗性协变量选择联系起来，该参数限制了条件协变量密度相对于一个固定的、未知的基础测度的比值。我们提出了 \textsc{Hedge-Cover}，一种信息论算法，对于具有有界伪维数的函数类，实现了次线性遗憾 $\widetilde{O}(\sqrt{\text{Pdim}(\mathcal{F}) \textsf{C}_{\textsf{cov}} T})$。该算法使用 \textsc{Hedge} 聚合一个精心构造的专家族，其先验将遗憾与一个一致选择器和一个目标函数之间的不一致次数联系起来。我们通过利用协变量平滑性来界定这一次数。这解决了 \cite{blanchard2025agnostic} 中提出的关于平滑在线回归问题极小极大最优自适应遗憾的开放问题。我们为线性预测器类建立了匹配的下界。下界的主要难点在于显式构造一个支持在相互正交的超平面上的具有挑战性的序列协变量分布。这一构造可能具有独立的技术意义。最后，我们重新审视了良好设定情形，并量化了响应噪声的影响。对于条件 $\nu^2$-次高斯响应，我们通过证明对于VC维为 $d$ 的函数类，极小极大期望遗憾为 $\Omega((1\vee \nu)\sqrt{(\textsf{C}_{\textsf{cov}}-1)dT})$，扩展了在可实现响应下的现有下界。相应的ERM上界在 $\nu$ 上匹配了这一依赖性。
英文摘要：
We study smoothed online prediction with bounded adversarial responses. This widely studied framework bridges i.i.d. sampling and adversarial covariate selection through a smoothness parameter $\textsf{C}_{\textsf{cov}}$, which bounds conditional covariate densities relative to a fixed, unknown base measure. We propose \textsc{Hedge-Cover}, an information-theoretic algorithm that achieves sublinear regret $\widetilde{O}(\sqrt{\text{Pdim}(\mathcal{F}) \textsf{C}_{\textsf{cov}} T})$ for function classes with bounded pseudo-dimension. The algorithm aggregates a carefully constructed family of experts using \textsc{Hedge}, with a prior that links regret to the number of disagreements between a consistent selector and a target function. We bound this number by exploiting covariate smoothness. This answers an open problem posed in \cite{blanchard2025agnostic} on the minimax optimal adaptive regret of the smoothed online regression problem. We establish a matching lower bound for the class of linear predictors. The main intricacy of the lower bound lies in explicitly constructing a challenging sequential covariate distribution supported on mutually orthogonal hyperplanes. This construction may be of independent technical interest. Finally, we revisit the well-specified setting and quantify the effect of response noise. For conditionally $ν^2$-subGaussian responses, we extend the existing lower bound under realizable responses by showing that the minimax expected regret is $Ω((1\vee ν)\sqrt{(\textsf{C}_{\textsf{cov}}-1)dT})$ for a function class of VC dimension $d$. A corresponding upper bound for ERM matches this dependence on $ν$.
