arXiv 2402.142292024-02-22math.STcs.DScs.LGstat.TH

具有自选择偏差的样本高效线性回归

Sample-Efficient Linear Regression with Self-Selection Bias

Massachusetts Institute of Technology（麻省理工学院）

Jason Gaitonde, Elchanan Mossel

备注：: 40 pages

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文研究未知索引设定下带自选择偏差的线性回归问题，提出一种样本复杂度接近最优且时间复杂度显著改善的算法，在放宽的噪声假设下恢复权重向量，并推广了已有矩匹配方法的多项式保证。

摘要：

我们考虑在未知索引设定下具有自选择偏差的线性回归问题，该问题由 Cherapanamjeri、Daskalakis、Ilyas 和 Zampetakis 在近期工作 [STOC 2023] 中引入。在该模型中，我们观测到 $m$ 个独立同分布样本 $(\mathbf{x}_{\ell},z_{\ell})_{\ell=1}^m$，其中 $z_{\ell}=\max_{i\in [k]}\{\mathbf{x}_{\ell}^T\mathbf{w}_i+\eta_{i,\ell}\}$，但取得最大值的索引 $i_{\ell}$ 不可观测。这里假设 $\mathbf{x}_{\ell}$ 服从 $\mathcal{N}(0,I_n)$，噪声分布 $\mathbf{\eta}_{\ell}\sim \mathcal{D}$ 是中心化的且与 $\mathbf{x}_{\ell}$ 独立。我们提出了一种新颖的、在 $k$ 方面接近最优样本效率的算法，以多项式样本复杂度 $\tilde{O}(n)\cdot \mathsf{poly}(k,1/\varepsilon)$ 恢复 $\mathbf{w}_1,\ldots,\mathbf{w}_k\in \mathbb{R}^n$ 至加性 $\ell_2$-误差 $\varepsilon$，并显著改善了时间复杂度至 $\mathsf{poly}(n,k,1/\varepsilon)+O(\log(k)/\varepsilon)^{O(k)}$。当 $k=O(1)$ 时，我们的算法运行时间为 $\mathsf{poly}(n,1/\varepsilon)$，推广了 Cherapanamjeri 等人针对 $k=2$ 且已知 $\mathcal{D}=\mathcal{N}(0,I_k)$ 的显式矩匹配算法的多项式保证。我们的算法在显著放宽的噪声假设下仍然成功，因此也在相关设定——即加性噪声取在最大值之外的 max-linear regression 中成功。对于该问题，在 $\varepsilon$ 不太小的情况下，我们的算法在比 Ghosh、Pananjady、Guntuboyina 和 Ramchandran [IEEE Trans. Inf. Theory 2022] 的最新成果大得多的 $k$ 范围内保持高效，并且通过为现有局部收敛方法提供热启动，对任意 $\varepsilon$ 都能得到改进的算法。

英文摘要：

We consider the problem of linear regression with self-selection bias in the unknown-index setting, as introduced in recent work by Cherapanamjeri, Daskalakis, Ilyas, and Zampetakis [STOC 2023]. In this model, one observes $m$ i.i.d. samples $(\mathbf{x}_{\ell},z_{\ell})_{\ell=1}^m$ where $z_{\ell}=\max_{i\in [k]}\{\mathbf{x}_{\ell}^T\mathbf{w}_i+η_{i,\ell}\}$, but the maximizing index $i_{\ell}$ is unobserved. Here, the $\mathbf{x}_{\ell}$ are assumed to be $\mathcal{N}(0,I_n)$ and the noise distribution $\mathbfη_{\ell}\sim \mathcal{D}$ is centered and independent of $\mathbf{x}_{\ell}$. We provide a novel and near optimally sample-efficient (in terms of $k$) algorithm to recover $\mathbf{w}_1,\ldots,\mathbf{w}_k\in \mathbb{R}^n$ up to additive $\ell_2$-error $\varepsilon$ with polynomial sample complexity $\tilde{O}(n)\cdot \mathsf{poly}(k,1/\varepsilon)$ and significantly improved time complexity $\mathsf{poly}(n,k,1/\varepsilon)+O(\log(k)/\varepsilon)^{O(k)}$. When $k=O(1)$, our algorithm runs in $\mathsf{poly}(n,1/\varepsilon)$ time, generalizing the polynomial guarantee of an explicit moment matching algorithm of Cherapanamjeri, et al. for $k=2$ and when it is known that $\mathcal{D}=\mathcal{N}(0,I_k)$. Our algorithm succeeds under significantly relaxed noise assumptions, and therefore also succeeds in the related setting of max-linear regression where the added noise is taken outside the maximum. For this problem, our algorithm is efficient in a much larger range of $k$ than the state-of-the-art due to Ghosh, Pananjady, Guntuboyina, and Ramchandran [IEEE Trans. Inf. Theory 2022] for not too small $\varepsilon$, and leads to improved algorithms for any $\varepsilon$ by providing a warm start for existing local convergence methods.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

AI总结：

摘要：

英文摘要：

相关论文