PoPS：面向深度强化学习的策略剪枝与收缩方法
PoPS: Policy Pruning and Shrinking for Deep Reinforcement Learning
AI总结：
针对深度强化学习中DNN剪枝后性能下降的问题，提出基于迁移学习的迭代式策略剪枝收缩算法PoPS，在多基准环境中验证了其在保持高性能的同时压缩模型的效果，并开源了相关工具。
摘要：
近年来深度神经网络（DNN）在强化学习函数近似任务中取得成功，推动了深度强化学习（DRL）算法在机器人、计算机游戏、自然语言处理、计算机视觉、传感系统、无线网络等多个领域的发展。但DNN存在计算成本高、内存消耗大的缺陷，限制了DRL算法在硬件资源有限的系统中的应用。近年来，剪枝算法在减少分类任务中DNN的冗余性方面已展现出显著成效，但现有算法应用于DRL领域时会出现性能大幅下降的问题。本文针对DRL领域剪枝导致的性能下降问题，提出了首个有效解决方案，构建了名为Policy Pruning and Shrinking（PoPS，策略剪枝与收缩）的可行算法，能够在训练出高性能DRL模型的同时实现DNN的紧凑表示。该框架基于一种新颖的迭代式策略剪枝与收缩方法，在训练DRL模型时利用迁移学习的优势。我们通过广泛的实验研究，在常用的Cartpole、Lunar Lander、Pong和Pacman环境中验证了PoPS的优异性能。最后，我们开发了一款开源软件，供相关领域的研究人员和开发者使用。
英文摘要：
The recent success of deep neural networks (DNNs) for function approximation in reinforcement learning has triggered the development of Deep Reinforcement Learning (DRL) algorithms in various fields, such as robotics, computer games, natural language processing, computer vision, sensing systems, and wireless networking. Unfortunately, DNNs suffer from high computational cost and memory consumption, which limits the use of DRL algorithms in systems with limited hardware resources. In recent years, pruning algorithms have demonstrated considerable success in reducing the redundancy of DNNs in classification tasks. However, existing algorithms suffer from a significant performance reduction in the DRL domain. In this paper, we develop the first effective solution to the performance reduction problem of pruning in the DRL domain, and establish a working algorithm, named Policy Pruning and Shrinking (PoPS), to train DRL models with strong performance while achieving a compact representation of the DNN. The framework is based on a novel iterative policy pruning and shrinking method that leverages the power of transfer learning when training the DRL model. We present an extensive experimental study that demonstrates the strong performance of PoPS using the popular Cartpole, Lunar Lander, Pong, and Pacman environments. Finally, we develop an open source software for the benefit of researchers and developers in related fields.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
