发表机构
New Mexico Institute of Mining and Technology; The University of Texas at El Paso（新墨西哥矿业技术学院; 德克萨斯大学埃尔帕索分校）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
该研究提出SANSAC算法，在传统硬件上验证其性能与SAC近乎相当，证明基于脉冲神经网络的算法可用于复杂连续环境，为神经形态强化学习研究奠定了基础。
AI 中文摘要
过去十年，强化学习（RL）算法取得了长足进步，已应用于各类问题与控制任务。然而，将RL部署到神经形态硬件以完成连续控制任务的验证仍不充分，具体而言，在硬件特定优势显现前，用脉冲神经网络（SNN）替代传统演员网络是否会影响智能体性能尚不明确。本文对软 Actor-评论家算法（SAC）的最小化、神经形态可行的脉冲演员变体（即SANSAC）在传统硬件上进行了系统验证，为未来神经形态RL研究建立了基线。我们提出脉冲演员网络软 Actor-评论家算法（SANSAC），以解决RL框架在连续环境中的应用问题，该框架可在神经形态硬件上实现。我们在传统计算机中对比了传统SAC网络与SANSAC的性能，验证了SANSAC与SAC的性能近乎相当，同时分析了隐藏层维度的影响。研究结果表明，基于SNN的算法在复杂连续环境中具备可行性，且在传统计算机上的性能可与传统神经网络媲美，为进一步探索SNN在连续RL框架中的应用提供了基础。
英文摘要
Reinforcement learning (RL) algorithms have made strides over the past decade applying them to a wide range of problems and control tasks. However, the deployment of RL on neuromorphic hardware for continuous control tasks remains under-validated. Namely it is unclear whether replacing a conventional actor network with a spiking neural network (SNN) affects the performance of an agent before any hardware-specific benefits manifest. We provide a systematic validation of a minimal, neuromorphically viable spiking actor variant of Soft Actor-Critic (SAC) on conventional hardware, establishing a baseline for future neuromorphic RL research. In this paper, we propose the Spiking Actor Network Soft Actor Critic (SANSAC) to address the use of RL frameworks in continuous environments, designed as a framework that can be implemented on neuromorphic hardware. We compare a traditional Soft Actor Critic (SAC) network to SANSAC in a traditional computer. We demonstrate the near equivalent performance of SANSAC and SAC, while addressing the impact of hidden dimensions. Our results demonstrate the viability of SNN based algorithms in complex continuous environments, as well as competitive performance to traditional neural networks in traditional computers, providing a basis to continue exploring the use of SNNs in continuous RL frameworks.
Journal refICLR 2026 2nd Workshop on World Models
