面向异构平台的深度强化学习软硬件协同优化工具包
A Software-Hardware Co-Optimized Toolkit for Deep Reinforcement Learning on Heterogeneous Platforms
AI总结:
提出PEARL工具包,通过硬件无关训练协议、调度优化和自动任务分配,在异构平台上提升DRL性能,吞吐量提升达2.1倍,能效提升达3.4倍。
摘要:
深度强化学习(DRL)在各种人工智能应用中至关重要。DRL算法包含多种计算内核,这些内核可能无法在单一同构架构上同时得到优化。然而,即使存在可用的异构架构,由于现代数据中心所采用的硬件和编程模型的复杂性,优化DRL性能仍然是一项挑战。为了解决这一问题,我们提出了PEARL,一个用于在由通用处理器(CPU)和加速器(GPU、FPGA)组成的异构平台上构建并行DRL系统的工具包。我们的创新包括:1. 一种与底层硬件无关的通用训练协议,使得实现能够在各种处理器和加速器之间可移植。2. 在该协议中纳入DRL特有的调度优化,促进并行训练并提升整体系统性能。3. 提供高级API,以便使用该工具包进行高效开发。4. 通过性能估计自动优化DRL任务到设备的分配,支持包括吞吐量和能效在内的多种优化指标。我们通过在两个不同的异构平台上使用两种广泛使用的DRL算法DQN和DDPG进行实验来展示我们的工具包。生成的实现相比最先进的CPU-GPU平台库,吞吐量提升最高达2.1倍,能效提升最高达3.4倍。
英文摘要:
Deep Reinforcement Learning (DRL) is vital in various AI applications. DRL algorithms comprise diverse compute kernels, which may not be simultaneously optimized using a homogeneous architecture. However, even with available heterogeneous architectures, optimizing DRL performance remains a challenge due to the complexity of hardware and programming models employed in modern data centers. To address this, we introduce PEARL, a toolkit for composing parallel DRL systems on heterogeneous platforms consisting of general-purpose processors (CPUs) and accelerators (GPUs, FPGAs). Our innovations include: 1. A general training protocol agnostic of the underlying hardware, enabling portable implementations across various processors and accelerators. 2. Incorporation of DRL-specific scheduling optimizations within the protocol, facilitating parallelized training and enhancing the overall system performance. 3. High-level API for productive development using the toolkit. 4. Automatic optimization of DRL task-to-device assignments through performance estimation, supporting various optimization metrics including throughput and power efficiency. We showcase our toolkit through experimentation with two widely used DRL algorithms, DQN and DDPG, on two diverse heterogeneous platforms. The generated implementations outperform state-of-the-art libraries for CPU-GPU platforms by throughput improvements of up to 2.1$\times$ and power efficiency improvements of up to 3.4$\times$.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。