arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1605.01636cs.LG
收藏

深度网络的最大稀疏性?

Maximal Sparsity with Deep Networks?

Microsoft Research(微软研究院)

Bo Xin, Yizhou Wang, Wen Gao, David Wipf

摘要:

许多稀疏估计算法的迭代由与阈值非线性级联的固定线性滤波器组成,其共同类似于典型的神经网络层。因此,一个冗长的算法迭代序列可以被视为一个具有共享的、手工制作的层权重的深度网络。因此,很自然地检查学习的网络模型可以作为传统稀疏估计的可行替代品的程度,在这些领域中,有充足的训练数据可用。虽然减少计算预算的可能性是显而易见的,当对层数施加上限时,我们的工作主要集中在估计精度上。特别是,它是众所周知的,当一个信号字典具有相干列,量化一个大的RIP常数,然后最听话的迭代算法是无法找到最大的稀疏表示。相比之下,我们从理论上和经验上证明了经过训练的深度网络在现有方法失败的情况下恢复最小$\ell_0$范数表示的潜力。由此产生的系统部署在一个实际的光度立体估计问题,其目标是删除稀疏离群值,可以破坏从3D场景的表面法线的估计。

英文摘要:

The iterations of many sparse estimation algorithms are comprised of a fixed linear filter cascaded with a thresholding nonlinearity, which collectively resemble a typical neural network layer. Consequently, a lengthy sequence of algorithm iterations can be viewed as a deep network with shared, hand-crafted layer weights. It is therefore quite natural to examine the degree to which a learned network model might act as a viable surrogate for traditional sparse estimation in domains where ample training data is available. While the possibility of a reduced computational budget is readily apparent when a ceiling is imposed on the number of layers, our work primarily focuses on estimation accuracy. In particular, it is well-known that when a signal dictionary has coherent columns, as quantified by a large RIP constant, then most tractable iterative algorithms are unable to find maximally sparse representations. In contrast, we demonstrate both theoretically and empirically the potential for a trained deep network to recover minimal $\ell_0$-norm representations in regimes where existing methods fail. The resulting system is deployed on a practical photometric stereo estimation problem, where the goal is to remove sparse outliers that can disrupt the estimation of surface normals from a 3D scene.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑