Asymmetry in the synaptic interactions between neurons plays a crucial role in determining the memory storage and retrieval properties of recurrent neural networks. In this work, we analyze the problem of storing random memories in a network of neurons connected by a synaptic matrix with a definite degree of asymmetry. We study the corresponding satisfiability and clustering transitions in the space of solutions of the constraint satisfaction problem associated with finding synaptic matrices given the memories. We find, besides the usual SAT/UNSAT transition at a critical number of memories to store in the network, an additional transition for very asymmetric matrices, where the competing constraints (definite asymmetry vs. memories storage) induce enough frustration in the problem to make it impossible to solve. This finding is particularly striking in the case of a single memory to store, where no quenched disorder is present in the system.
Neural Langevin Machine: a local asymmetric learning rule can be creative
PMI Lab, School of Physics, Sun Yat-sen University(物理系,中山大学,PMI实验室);Guangdong Provincial Key Laboratory of Magnetoelectric Physics and Devices, Sun Yat-sen University(磁电物理与器件广东省重点实验室,中山大学)
Embedding stochastic dynamics in latent manifolds of recurrent neural networks
Mathematical Institute, University of Oxford(牛津大学数学研究所);Centre for Eudaimonia and Human Flourishing, University of Oxford(牛津大学幸福与人类繁荣中心);Complexity Science Hub, Vienna(维也纳复杂科学中心)