arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1709.01112cs.ITmath.ITstat.ML
收藏

基于Laplace技术的稀疏恢复最优深度神经网络

Optimal deep neural networks for sparse recovery via Laplace techniques

Steffen Limmer, Slawomir Stanczak

摘要:

本文介绍了拉普拉斯技术设计的神经网络,从压缩测量估计单纯形约束稀疏向量的目标。为此,我们重铸的MMSE估计(w.r.t.预定义的均匀输入分布)作为计算某个多面体的质心的问题,该多面体由单纯形和由测量确定的仿射子空间的相交产生。由于特定的结构,它示出的质心可以计算分析通过扩展最近的结果,便于体积计算的多面体通过拉普拉斯变换。本文的一个主要观点是,所需的体积和质心计算可以通过经典的深度神经网络来执行,该深度神经网络包括阈值函数、整流线性(ReLU)和整流多项式(ReP)激活函数。所提出的用于稀疏恢复的深度神经网络的构造是完全解析的,因此不需要耗时的训练过程。此外,我们表明,在我们的建设层的数量是等于测量的数量,这可能使新的低延迟稀疏恢复算法的一个更大的一类信号比本文中假设的。为了评估所提出的均匀输入分布的适用性,我们展示了由两个标准数据集生成的软分类向量样本的恢复性能。由于体积和质心计算都是已知的计算困难,网络宽度在最坏情况下呈指数增长。然而,它可以减少诱导稀疏连接的神经网络通过一个非常适合的基础的仿射子空间。最后,所提出的分析构造可以用作可行的初始化,以使用手头的特定输入数据集进行进一步优化和训练。

英文摘要:

This paper introduces Laplace techniques for designing a neural network, with the goal of estimating simplex-constraint sparse vectors from compressed measurements. To this end, we recast the problem of MMSE estimation (w.r.t. a pre-defined uniform input distribution) as the problem of computing the centroid of some polytope that results from the intersection of the simplex and an affine subspace determined by the measurements. Owing to the specific structure, it is shown that the centroid can be computed analytically by extending a recent result that facilitates the volume computation of polytopes via Laplace transformations. A main insight of this paper is that the desired volume and centroid computations can be performed by a classical deep neural network comprising threshold functions, rectified linear (ReLU) and rectified polynomial (ReP) activation functions. The proposed construction of a deep neural network for sparse recovery is completely analytic so that time-consuming training procedures are not necessary. Furthermore, we show that the number of layers in our construction is equal to the number of measurements which might enable novel low-latency sparse recovery algorithms for a larger class of signals than that assumed in this paper. To assess the applicability of the proposed uniform input distribution, we showcase the recovery performance on samples that are soft-classification vectors generated by two standard datasets. As both volume and centroid computation are known to be computationally hard, the network width grows exponentially in the worst-case. It can be, however, decreased by inducing sparse connectivity in the neural network via a well-suited basis of the affine subspace. Finally, the presented analytical construction may serve as a viable initialization to be further optimized and trained using particular input datasets at hand.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑