arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1602.05719cs.LGcs.DScs.ITmath.ITmath.PR
收藏

ER-SpUD字典学习算法的改进分析

An improved analysis of the ER-SpUD dictionary learning algorithm

Harvard University(哈佛大学)

Jarosław Błasiok, Jelani Nelson

摘要:

在“字典学习”中,我们观察到一些$Y\in\mathbb{R}^{n\times p}$、$A \in\mathbb{R}^{m\times n}$和$X\in\mathbb{R}^{m\times p}$的$Y = AX + E$。观察到基质$Y$,$A, X, E$未知。这里$E$是小范数的“噪声”,$X$是列稀疏的。矩阵$A$被称为{ \em字典},其列被称为{ \em原子}。然后,给定一些少量的样本$p$,即\列的$Y$,目标是学习字典$A$,最多可以小错误,以及$X$。其动机是,在许多应用程序中,当数据由“正确”字典$A$中的原子表示时,预计数据是稀疏的(例如,\ Haar小波基中的图像),目标是从数据中学习$A$,然后将其用于其他应用程序。最近,[SWW 12]提出了字典学习算法ER-SpUD,当$E = 0$和$m = n$时具有可证明的保证。他们表明,如果$X$具有独立的条目,对于$1 \lesssim s \lesssim \sqrt{n}$,每列预期$s$为非零,并且非零条目为亚高斯,则对于$p\gtrsim n^2\log^2 n$,具有高概率的ER-SpUD输出矩阵$A', X'$,其等于$A, X$直到排列和缩放列(分别为\行)的$A$(分别为\ $X$)。他们证明了$p\gtrsim n\log n$足够,他们表明,当$s \simeq 1$时,这是{ \em any}算法成功的理论上所必需的信息。后来在[LV 15]中取得了重大进展。我们表明,对于ER-SpUD的轻微变化,$p\gtrsim n\log(n/δ)$样本足以成功恢复概率$1-δ$。我们还表明,对于未经修改的ER-SpUD,$p\gtrsim n^{1.99}$样本需要学习$A, X$多项式小的成功概率。这解决了[SWW 12]的主要猜想,并与[LV 15]的主要结果相矛盾,[LV 15]声称$p\gtrsim n\log^4 n$保证whp成功。

英文摘要:

In "dictionary learning" we observe $Y = AX + E$ for some $Y\in\mathbb{R}^{n\times p}$, $A \in\mathbb{R}^{m\times n}$, and $X\in\mathbb{R}^{m\times p}$. The matrix $Y$ is observed, and $A, X, E$ are unknown. Here $E$ is "noise" of small norm, and $X$ is column-wise sparse. The matrix $A$ is referred to as a {\em dictionary}, and its columns as {\em atoms}. Then, given some small number $p$ of samples, i.e.\ columns of $Y$, the goal is to learn the dictionary $A$ up to small error, as well as $X$. The motivation is that in many applications data is expected to sparse when represented by atoms in the "right" dictionary $A$ (e.g.\ images in the Haar wavelet basis), and the goal is to learn $A$ from the data to then use it for other applications. Recently, [SWW12] proposed the dictionary learning algorithm ER-SpUD with provable guarantees when $E = 0$ and $m = n$. They showed if $X$ has independent entries with an expected $s$ non-zeroes per column for $1 \lesssim s \lesssim \sqrt{n}$, and with non-zero entries being subgaussian, then for $p\gtrsim n^2\log^2 n$ with high probability ER-SpUD outputs matrices $A', X'$ which equal $A, X$ up to permuting and scaling columns (resp.\ rows) of $A$ (resp.\ $X$). They conjectured $p\gtrsim n\log n$ suffices, which they showed was information theoretically necessary for {\em any} algorithm to succeed when $s \simeq 1$. Significant progress was later obtained in [LV15]. We show that for a slight variant of ER-SpUD, $p\gtrsim n\log(n/δ)$ samples suffice for successful recovery with probability $1-δ$. We also show that for the unmodified ER-SpUD, $p\gtrsim n^{1.99}$ samples are required even to learn $A, X$ with polynomially small success probability. This resolves the main conjecture of [SWW12], and contradicts the main result of [LV15], which claimed that $p\gtrsim n\log^4 n$ guarantees success whp.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑