发表机构
Drexel University; Los Alamos National Laboratory; Northwestern University（德雷塞尔大学; 洛斯阿拉莫斯国家实验室; 西北大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文提出并实现了基于Loihi 2的全片上闭环Q学习强化学习代理，通过嵌入式CartPole仿真验证，在相同成功率下执行时间减半、功耗降低两个数量级，展示了神经形态硬件在节能实时嵌入式AI中的应用潜力。
AI 中文摘要
随着AI模型规模和使用的增长，其能源需求急剧增加，引发了可持续性和经济方面的担忧。受大脑能效启发的神经形态硬件，通过提供低功耗、快速处理的替代方案来应对传统计算面临的这一挑战。此类硬件特别适用于资源受限环境中的控制系统，这些系统最好通过强化学习（RL）进行训练。本贡献介绍了全片上、闭环Loihi 2 RL代理的设计与实现。我们的神经形态电路包括一个完全嵌入的Q学习算法和Loihi 2上CartPole-v0环境的片上仿真。我们的Q学习算法训练了与CPU实现相同数量的成功代理，但执行时间仅为其一半，动态功耗低两个数量级。这些发现证明了RL在神经形态硬件上的可行性，并突显了其构建节能、实时、嵌入式AI系统的前景。
英文摘要
As AI models grow in size and usage, their energy demands increase dramatically, raising sustainability and economic concerns. Neuromorphic hardware, inspired by the energy efficiency of the brain, seeks to address this challenge by offering low-power, fast-processing alternatives to conventional computing. Such hardware is particularly well-suited to control systems deployed in resource-constrained environments, which are best trained via reinforcement learning (RL). This contribution presents the design and implementation of a fully on-chip, closed-loop Loihi 2 RL agent. Our neuromorphic circuit consists of a fully embedded Q-learning algorithm and an on-chip simulation of the CartPole-v0 environment on Loihi 2. Our Q-learning algorithm trained the same number of successful agents as the CPU implementation in only half the execution time and with two orders of magnitude less dynamic power. These findings demonstrate the viability of RL on neuromorphic hardware and highlight its promise for building energy-efficient, real-time, embedded AI systems.
Comments10 pages, 4 figures, 2 tables
