arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1805.08952cs.LGstat.ML
收藏

基于动态神经网络的词典学习

Dictionary Learning by Dynamical Neural Networks

Intel Corporation(英特尔公司)

Tsung-Han Lin, Ping Tak Peter Tang

摘要:

动态神经网络由一组相互连接的神经元组成,这些神经元随时间连续交互。它可以表现出计算特性,在这个意义上,动力系统的演化和/或在相关状态空间中的极限点可以对应于某些数学优化或学习问题的数值解。这样的计算系统特别有吸引力,因为它可以被映射到大规模并行计算机体系结构以获得功率和吞吐量效率,特别是如果每个神经元可以单独依赖于局部信息(即,本地存储器)。然而,从动态网络的各种状态导出梯度,同时符合最后一个约束,是具有挑战性的。我们表明,通过结合自上而下的反馈和对比学习的思想,可以构建一个动态网络来解决l1-最小化字典学习问题,并证明学习的真实梯度是可计算的单个神经元。使用尖峰神经元来构建我们的动态网络,我们提出了一个学习过程,其严格的数学分析,并对几个字典学习问题的数值结果。

英文摘要:

A dynamical neural network consists of a set of interconnected neurons that interact over time continuously. It can exhibit computational properties in the sense that the dynamical system's evolution and/or limit points in the associated state space can correspond to numerical solutions to certain mathematical optimization or learning problems. Such a computational system is particularly attractive in that it can be mapped to a massively parallel computer architecture for power and throughput efficiency, especially if each neuron can rely solely on local information (i.e., local memory). Deriving gradients from the dynamical network's various states while conforming to this last constraint, however, is challenging. We show that by combining ideas of top-down feedback and contrastive learning, a dynamical network for solving the l1-minimizing dictionary learning problem can be constructed, and the true gradients for learning are provably computable by individual neurons. Using spiking neurons to construct our dynamical network, we present a learning process, its rigorous mathematical analysis, and numerical results on several dictionary learning problems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑