离群值鲁棒的加性矩阵分解
Outlier-robust additive matrix decomposition
AI总结:
本文研究了标签被破坏的最小二乘迹回归,提出基于“排序”Huber损失的可处理估计量,在处理加性矩阵分解、标签污染与噪声交互时达到近最优率。
摘要:
我们研究了最小二乘迹回归,其中参数是一个 $r$-低秩矩阵和一个 $s$-稀疏矩阵之和,且有 $ε$ 比例的标签被破坏。对于次高斯分布和特征相关噪声,我们强调了三个所需的设计性质,每个性质源自不同的过程不等式:“乘积过程不等式”、“Chevet不等式”和“乘数过程不等式”。这些性质同时处理加性分解、标签污染和设计-噪声交互。它们意味着一个可处理估计量关于低秩和稀疏成分的有效维度 $d_{eff,r}$ 和 $d_{eff,s}$、$ε$ 以及失败概率 $δ$ 的近最优性。近最优率为 $\mathsf{r}(n,d_{eff,r}) + \mathsf{r}(n,d_{eff,s}) + \sqrt{(1+\log(1/δ))/n} + ε\log(1/ε)$,其中 $\mathsf{r}(n,d_{eff,r})+\mathsf{r}(n,d_{eff,s})$ 是无污染时的平均最优率。我们的估计量自适应于 $(s,r,ε,δ)$,对于固定绝对常数 $c>0$,它以 $1-δ$ 的概率在所有 $δ\ge\exp(-cn)$ 上一致达到上述率。若无矩阵分解,我们的分析还为适应噪声方差的鲁棒估计量提供了最优界。我们的估计量基于 Huber 损失的“排序”版本。我们展示了与理论相符的模拟。特别地,它揭示了“排序” Huber 损失相对于经典 Huber 损失的优越性。
英文摘要:
We study least-squares trace regression when the parameter is the sum of a $r$-low-rank matrix and a $s$-sparse matrix and a fraction $ε$ of the labels is corrupted. For subgaussian distributions and feature-dependent noise, we highlight three needed design properties, each one derived from a different process inequality: a "product process inequality", "Chevet's inequality" and a "multiplier process inequality". These properties handle, simultaneously, additive decomposition, label contamination and design-noise interaction. They imply the near-optimality of a tractable estimator with respect to the effective dimensions $d_{eff,r}$ and $d_{eff,s}$ of the low-rank and sparse components, $ε$ and the failure probability $δ$. The near-optimal rate is $\mathsf{r}(n,d_{eff,r}) + \mathsf{r}(n,d_{eff,s}) + \sqrt{(1+\log(1/δ))/n} + ε\log(1/ε)$, where $\mathsf{r}(n,d_{eff,r})+\mathsf{r}(n,d_{eff,s})$ is the optimal rate in average with no contamination. Our estimator is adaptive to $(s,r,ε,δ)$ and, for fixed absolute constant $c>0$, it attains the mentioned rate with probability $1-δ$ uniformly over all $δ\ge\exp(-cn)$. Without matrix decomposition, our analysis also entails optimal bounds for a robust estimator adapted to the noise variance. Our estimators are based on "sorted" versions of Huber's loss. We present simulations matching the theory. In particular, it reveals the superiority of "sorted" Huber's losses over the classical Huber's loss.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。