Learning in networks of binary synapses is known to be an NP-complete problem. A combined stochastic local search strategy in the synaptic weight space is constructed to further improve the learning performance of a single random walker. We apply two correlated random walkers guided by their Hamming distance and associated energy costs (the number of unlearned patterns) to learn a same large set of patterns. Each walker first learns a small part of the whole pattern set (partially different for both walkers but with the same amount of patterns) and then both walkers explore their respective weight spaces cooperatively to find a solution to classify the whole pattern set correctly. The desired solutions locate at the common parts of weight spaces explored by these two walkers. The efficiency of this combined strategy is supported by our extensive numerical simulations and the typical Hamming distance as well as energy cost is estimated by an annealed computation.
On the role of synaptic stochasticity in training low-precision neural networks
Bocconi Institute for Data Science and Analytics, Bocconi University(博科尼大学博科尼数据科学与分析研究所);Italian Institute for Genomic Medicine(意大利基因组医学研究所);Istituto Nazionale di Fisica Nucleare(意大利国家核物理研究所);Politecnico di Torino(都灵理工大学);Radboud University Nijmegen(内梅亨大学);Donders Institute for Brain, Cognition and Behaviour(唐德斯大脑、认知与行为研究所);International Centre for Theoretical Physics(国际理论物理中心)
Biologically Plausible Sequence Learning with Spiking Neural Networks
National University of Singapore(新加坡国立大学);Singapore University of Technology and Design(新加坡科技设计大学);Chulalongkorn University(朱拉隆功大学);Awecom, Inc(奥威康公司);University of California, Berkeley(加州大学伯克利分校)