arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1906.01668cs.LGcs.NEstat.ML
收藏

面向特定任务动态学习的神经元结构优化

Neuromorphic Architecture Optimization for Task-Specific Dynamic Learning

Argonne National Laboratory(阿贡国家实验室)

Sandeep Madireddy, Angel Yanguas-Gil, Prasanna Balaprakash

摘要:

实时的学习和适应的能力是生物系统的核心特征。展示这种多功能性的神经形态架构可以大大增强我们在边缘有效处理信息的能力。然而,一个关键的挑战是了解哪些学习规则最适合特定的任务,以及如何微调相关的超参数。在这项工作中,我们引入了一个概念框架,其中的学习过程被集成到网络本身。这使我们能够将元学习视为数学优化问题。我们采用DeepHyper,一种可扩展的异步基于模型的搜索,同时优化元学习规则及其超参数的选择。我们展示了我们的方法与两个不同的数据集,MNIST和FashionMNIST,使用的网络架构的灵感来自昆虫大脑的学习中心。我们的研究结果表明,即使在类似的任务中,最佳的学习规则也可能是依赖于网络的。这种依赖性证明了在学习算法中引入通用性和灵活性的重要性。它还阐明了昆虫神经科学中的实验发现,这些发现表明昆虫蘑菇体内的学习规则具有异质性。

英文摘要:

The ability to learn and adapt in real time is a central feature of biological systems. Neuromorphic architectures demonstrating such versatility can greatly enhance our ability to efficiently process information at the edge. A key challenge, however, is to understand which learning rules are best suited for specific tasks and how the relevant hyperparameters can be fine-tuned. In this work, we introduce a conceptual framework in which the learning process is integrated into the network itself. This allows us to cast meta-learning as a mathematical optimization problem. We employ DeepHyper, a scalable, asynchronous model-based search, to simultaneously optimize the choice of meta-learning rules and their hyperparameters. We demonstrate our approach with two different datasets, MNIST and FashionMNIST, using a network architecture inspired by the learning center of the insect brain. Our results show that optimal learning rules can be dataset-dependent even within similar tasks. This dependency demonstrates the importance of introducing versatility and flexibility in the learning algorithms. It also illuminates experimental findings in insect neuroscience that have shown a heterogeneity of learning rules within the insect mushroom body.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑