arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2306.03530cs.LGcs.AIcs.RO
收藏

RLtools:一个用于连续控制的快速、可移植深度强化学习库

RLtools: A Fast, Portable Deep Reinforcement Learning Library for Continuous Control

New York University(纽约大学);Technology Innovation Institute(技术创新研究院)

Jonas Eschmann, Dario Albani, Giuseppe Loianno

AI总结:

RLtools是一个无依赖、仅头文件的纯C++深度强化学习库,通过算法与仿真环境紧密集成实现比流行框架快76倍的训练速度,并首次在微控制器上直接训练深度RL算法,催生了TinyRL领域。

摘要:

深度强化学习(RL)能够在多个领域产生有能力的智能体和控制策略,但通常受到过长的训练时间困扰。此外,在连续控制问题中,由于现有库缺乏实时保证和可移植性,学习到的策略在真实世界嵌入式设备上的适用性受到限制。为了解决这些挑战,我们提出了RLtools,这是一个无依赖、仅头文件、纯C++的深度监督学习和强化学习库。其新颖的架构使RLtools能够用于各种平台,从HPC集群、工作站和笔记本电脑到智能手机、智能手表和微控制器。具体而言,由于RL算法与仿真环境的紧密集成,RLtools解决流行RL问题的速度比其他流行RL框架快达76倍。我们还在多种微控制器上对推理进行了基准测试,并表明在大多数情况下,我们的优化实现是迄今为止最快的。最后,RLtools首次实现了直接在微控制器上训练深度RL算法的演示,从而催生了TinyRL领域。源代码、文档和实时演示可通过我们的项目页面 https://rl.tools 获取。

英文摘要:

Deep Reinforcement Learning (RL) can yield capable agents and control policies in several domains but is commonly plagued by prohibitively long training times. Additionally, in the case of continuous control problems, the applicability of learned policies on real-world embedded devices is limited due to the lack of real-time guarantees and portability of existing libraries. To address these challenges, we present RLtools, a dependency-free, header-only, pure C++ library for deep supervised and reinforcement learning. Its novel architecture allows RLtools to be used on a wide variety of platforms, from HPC clusters over workstations and laptops to smartphones, smartwatches, and microcontrollers. Specifically, due to the tight integration of the RL algorithms with simulation environments, RLtools can solve popular RL problems up to 76 times faster than other popular RL frameworks. We also benchmark the inference on a diverse set of microcontrollers and show that in most cases our optimized implementation is by far the fastest. Finally, RLtools enables the first-ever demonstration of training a deep RL algorithm directly on a microcontroller, giving rise to the field of TinyRL. The source code as well as documentation and live demos are available through our project page at https://rl.tools.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑