arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1708.08705cs.CVcs.LGstat.ML
收藏

多层卷积稀疏建模:追踪和字典学习

Multi-Layer Convolutional Sparse Modeling: Pursuit and Dictionary Learning

Technion-Israel Institute of Technology(以色列理工学院);Stanford University(斯坦福大学)

Jeremias Sulam, Vardan Papyan, Yaniv Romano, Michael Elad

摘要:

最近提出的多层卷积稀疏编码(ML-CSC)模型,由级联的卷积稀疏层,提供了一个新的解释卷积神经网络(CNN)。在这个框架下,CNN中的前向传递的计算相当于一个追踪算法,旨在从给定的输入信号中估计嵌套的稀疏表示向量(或特征图)。尽管已经成为CNN和稀疏建模之间的关键联系,但仍然缺乏对ML-CSC的更深入理解:没有追踪算法可以精确地服务于这个模型,也没有条件保证非空模型。虽然可以容易地获得近似满足ML-CSC约束的信号,但仍然不清楚如何简单地从模型中采样,更重要的是,如何从真实数据中训练卷积滤波器。在这项工作中,我们提出了一个健全的追求算法的ML-CSC模型,采用投影方法。我们提供了新的和改进的边界上的稳定性的解决方案,这样的追求,我们分析了不同的实际替代方案,以实现这在实践中。我们证明了滤波器的训练对于模型中的非平凡信号至关重要,并且我们推导出一种在线算法来从真实数据中学习字典,有效地产生级联稀疏卷积层。最后,但并非最不重要的是,我们证明了ML-CSC模型在无监督环境中的几个应用程序的适用性,提供了有竞争力的结果。我们的工作代表了矩阵分解,稀疏字典学习和稀疏自动编码器之间的桥梁,我们详细分析了这些连接。

英文摘要:

The recently proposed Multi-Layer Convolutional Sparse Coding (ML-CSC) model, consisting of a cascade of convolutional sparse layers, provides a new interpretation of Convolutional Neural Networks (CNNs). Under this framework, the computation of the forward pass in a CNN is equivalent to a pursuit algorithm aiming to estimate the nested sparse representation vectors -- or feature maps -- from a given input signal. Despite having served as a pivotal connection between CNNs and sparse modeling, a deeper understanding of the ML-CSC is still lacking: there are no pursuit algorithms that can serve this model exactly, nor are there conditions to guarantee a non-empty model. While one can easily obtain signals that approximately satisfy the ML-CSC constraints, it remains unclear how to simply sample from the model and, more importantly, how one can train the convolutional filters from real data. In this work, we propose a sound pursuit algorithm for the ML-CSC model by adopting a projection approach. We provide new and improved bounds on the stability of the solution of such pursuit and we analyze different practical alternatives to implement this in practice. We show that the training of the filters is essential to allow for non-trivial signals in the model, and we derive an online algorithm to learn the dictionaries from real data, effectively resulting in cascaded sparse convolutional layers. Last, but not least, we demonstrate the applicability of the ML-CSC model for several applications in an unsupervised setting, providing competitive results. Our work represents a bridge between matrix factorization, sparse dictionary learning and sparse auto-encoders, and we analyze these connections in detail.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑