arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1106.5826cs.LGmath.STstat.MLstat.TH
收藏

多元稀疏回归的Dirty模型

A Dirty Model for Multiple Sparse Regression

The University of Texas at Austin(德克萨斯大学奥斯汀分校)

Ali Jalali, Pradeep Ravikumar, Sujay Sanghavi

摘要:

稀疏线性回归-从线性测量中找到未知向量-现在已知可以通过LASSO等方法使用比变量少的样本。我们考虑多稀疏线性回归问题,其中几个相关的向量-部分共享的支持集-必须恢复。在这种情况下,一个自然的问题是是否可以使用共享来进一步减少所需的样本总数。最近的一系列研究已经研究了使用\ell_1 / \ell_q范数块正则化(q>1)来解决此类问题;然而,这些方法实际上在样本复杂度方面的表现更差-相对于单独解决每个问题而忽略共享-取决于共享的水平。我们提出了一种新的多元稀疏线性回归方法,该方法可以在支持度和参数重叠存在时利用支持度和参数重叠,但在不存在时不支付罚款。一个非常简单的想法:我们将参数分解为两个分量,并以不同的方式正则化它们。我们在理论上和经验表明,我们的方法严格和明显优于\ell_1或\ell_1 / \ell_q方法,在整个范围内可能的重叠(除了在边界的情况下,我们匹配的最佳方法)。我们还提供了理论保证,该方法在高维尺度下表现良好。

英文摘要:

Sparse linear regression -- finding an unknown vector from linear measurements -- is now known to be possible with fewer samples than variables, via methods like the LASSO. We consider the multiple sparse linear regression problem, where several related vectors -- with partially shared support sets -- have to be recovered. A natural question in this setting is whether one can use the sharing to further decrease the overall number of samples required. A line of recent research has studied the use of \ell_1/\ell_q norm block-regularizations with q>1 for such problems; however these could actually perform worse in sample complexity -- vis a vis solving each problem separately ignoring sharing -- depending on the level of sharing. We present a new method for multiple sparse linear regression that can leverage support and parameter overlap when it exists, but not pay a penalty when it does not. A very simple idea: we decompose the parameters into two components and regularize these differently. We show both theoretically and empirically, our method strictly and noticeably outperforms both \ell_1 or \ell_1/\ell_q methods, over the entire range of possible overlaps (except at boundary cases, where we match the best method). We also provide theoretical guarantees that the method performs well under high-dimensional scaling.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑