arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2503.23972cs.LGcs.AI
收藏

基于噪声的奖励调节学习

Noise-based reward-modulated learning

Jesús García Fernández, Nasir Ahmad, Marcel van Gerven

AI总结:

本文提出基于噪声的奖励调节学习,通过生物启发式方法实现节能且适应性强的神经形态AI,展示了其在多层网络中的优越性能和可扩展性。

摘要:

追求节能和适应性的人工智能(AI)已将神经形态计算定位为传统计算的有希望的替代方案。然而,在这些平台上实现学习需要优先考虑局部信息同时能够有效分配信用的技术。本文提出了一种基于噪声的奖励调节学习(NRL),这是一种新颖的突触可塑性规则,通过数学方法将强化学习和基于梯度的优化统一起来,具有生物启发式的局部更新。NRL通过随机神经活动近似精确梯度,从而解决计算瓶颈问题,将生物和神经形态基质的固有噪声转化为功能性资源。受生物学习启发,我们的方法将奖励预测误差作为优化目标,以生成越来越有利的行为,并利用资格痕迹来促进回顾性信用分配。在强化任务上的实验验证,包括即时和延迟奖励,显示NRL在性能上与使用反向传播优化的基线相当,尽管收敛速度较慢,但在多层网络中相比奖励调节赫布学习(RMHL),最显著的类似方法,表现出明显更优越的性能和可扩展性。尽管在简单架构上测试,结果凸显了噪声驱动、脑启发式学习在低功耗适应系统中的潜力,特别是在具有局部性约束的计算基质中。NRL提供了一个理论基础坚实的范式,非常适合下一代神经形态AI的事件驱动特性。

英文摘要:

The pursuit of energy-efficient and adaptive artificial intelligence (AI) has positioned neuromorphic computing as a promising alternative to conventional computing. However, achieving learning on these platforms requires techniques that prioritize local information while enabling effective credit assignment. Here, we propose noise-based reward-modulated learning (NRL), a novel synaptic plasticity rule that mathematically unifies reinforcement learning and gradient-based optimization with biologically-inspired local updates. NRL addresses the computational bottleneck of exact gradients by approximating them through stochastic neural activity, transforming the inherent noise of biological and neuromorphic substrates into a functional resource. Drawing inspiration from biological learning, our method uses reward prediction errors as its optimization target to generate increasingly advantageous behavior, and eligibility traces to facilitate retrospective credit assignment. Experimental validation on reinforcement tasks, featuring immediate and delayed rewards, shows that NRL achieves performance comparable to baselines optimized using backpropagation, although with slower convergence, while showing significantly superior performance and scalability in multi-layer networks compared to reward-modulated Hebbian learning (RMHL), the most prominent similar approach. While tested on simple architectures, the results highlight the potential of noise-driven, brain-inspired learning for low-power adaptive systems, particularly in computing substrates with locality constraints. NRL offers a theoretically grounded paradigm well-suited for the event-driven characteristics of next-generation neuromorphic AI.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑