arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2504.13541cs.NEcs.AIcs.LGcs.RO
收藏

通过具有自适应任务切换策略的脉冲神经网络实现可扩展的多任务学习用于智能自主代理

Scalable Multi-Task Learning through Spiking Neural Networks with Adaptive Task-Switching Policy for Intelligent Autonomous Agents

eBRAIN Lab, New York University (NYU) Abu Dhabi(eBRAIN实验室,纽约大学(NYU)阿布扎比分校)

Rachmad Vidya Wicaksana Putra, Avaneesh Devkota, Muhammad Shafique

AI总结:

本文提出SwitchMT方法,通过自适应任务切换策略和深度脉冲Q网络实现高效的多任务学习,提升自主代理在多个Atari游戏中表现。

摘要:

在资源受限的自主代理上同时训练多个任务对于适应多样化的现实环境至关重要。最近的工作采用强化学习(RL)方法,但任务干扰仍导致多任务性能次优。最先进的工作利用脉冲神经网络(SNNs)改进基于RL的多任务学习并实现低功耗/能耗操作。然而,它们在训练期间依赖固定的任务切换间隔,限制了性能和可扩展性。为了解决这个问题,我们提出了SwitchMT,一种新颖的方法,采用自适应任务切换以实现有效、可扩展和同时的多任务学习。SwitchMT采用以下关键思想:(1)利用具有主动树突和对决结构的深度脉冲Q网络,利用任务特定的上下文信号创建专门的子网络;(2)设计一种自适应的任务切换策略,利用奖励和网络参数的内部动态。实验结果表明,SwitchMT在多个Atari游戏中(即Pong:-8.8,Breakout:5.6,Enduro:355.2)取得了竞争性的得分和更长的游戏回合,优于最先进的方法。这些结果也突显了SwitchMT方法在不增加网络复杂性的情况下解决任务干扰的有效性,使智能自主代理具备可扩展的多任务学习能力。

英文摘要:

Training resource-constrained autonomous agents on multiple tasks simultaneously is crucial for adapting to diverse real-world environments. Recent works employ reinforcement learning (RL) approach, but they still suffer from sub-optimal multi-task performance due to task interference. State-of-the-art works employ Spiking Neural Networks (SNNs) to improve RL-based multi-task learning and enable low-power/energy operations through network enhancements and spike-driven data stream processing. However, they rely on fixed task-switching intervals during its training, thus limiting its performance and scalability. To address this, we propose SwitchMT, a novel methodology that employs adaptive task-switching for effective, scalable, and simultaneous multi-task learning. SwitchMT employs the following key ideas: (1) leveraging a Deep Spiking Q-Network with active dendrites and dueling structure, that utilizes task-specific context signals to create specialized sub-networks; and (2) devising an adaptive task-switching policy that leverages both rewards and internal dynamics of the network parameters. Experimental results demonstrate that SwitchMT achieves competitive scores in multiple Atari games (i.e., Pong: -8.8, Breakout: 5.6, and Enduro: 355.2) and longer game episodes as compared to the state-of-the-art. These results also highlight the effectiveness of SwitchMT methodology in addressing task interference without increasing the network complexity, enabling intelligent autonomous agents with scalable multi-task learning capabilities.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑