高维统计量的可压缩分布
Compressible Distributions for High-dimensional Statistics
摘要：
我们开发了一种识别独立同分布（iid）实现是可压缩的概率分布的原则性方法，即，可以很好地近似为稀疏。我们专注于高斯随机欠定线性回归（GULR）的问题，压缩性是已知的，以确保成功的估计利用稀疏正则化。我们证明了许多分布围绕最大后验概率（MAP）解释稀疏正则估计实际上是不可压缩的，在大的问题大小的限制。一个亮点是拉普拉斯分布和$\ell^{1}$正则化估计器，如Lasso和Basis Pursuit去噪。为了建立这一结果，我们确定非平凡的欠采样区域在GULR简单的最小二乘解几乎肯定优于甲骨文稀疏的解决方案，当数据是从拉普拉斯分布。我们提供了简单的经验法则来描述类的可压缩（分别不可压缩）分布的基础上，他们的第二和第四时刻。广义高斯分布和广义帕累托分布作为具体性的运行示例。
英文摘要：
We develop a principled way of identifying probability distributions whose independent and identically distributed (iid) realizations are compressible, i.e., can be well-approximated as sparse. We focus on Gaussian random underdetermined linear regression (GULR) problems, where compressibility is known to ensure the success of estimators exploiting sparse regularization. We prove that many distributions revolving around maximum a posteriori (MAP) interpretation of sparse regularized estimators are in fact incompressible, in the limit of large problem sizes. A highlight is the Laplace distribution and $\ell^{1}$ regularized estimators such as the Lasso and Basis Pursuit denoising. To establish this result, we identify non-trivial undersampling regions in GULR where the simple least squares solution almost surely outperforms an oracle sparse solution, when the data is generated from the Laplace distribution. We provide simple rules of thumb to characterize classes of compressible (respectively incompressible) distributions based on their second and fourth moments. Generalized Gaussians and generalized Pareto distributions serve as running examples for concreteness.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
