arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2211.16385cs.ARcs.AIcs.LGcs.MA
收藏

微处理器设计空间探索的多智能体强化学习

Multi-Agent Reinforcement Learning for Microprocessor Design Space Exploration

Harvard University(哈佛大学);Google Research(谷歌研究院)

Srivatsan Krishnan, Natasha Jaques, Shayegan Omidshafiei, Dan Zhang, Izzeddin Gur, Vijay Janapa Reddi, Aleksandra Faust

摘要:

微处理器架构师越来越多地诉诸于特定领域的定制,以追求高性能和能源效率。随着系统复杂性的增长,跨多个子系统(例如,数据路径、不同层次结构中的存储器块、互连、编译器优化等)快速导致设计空间的组合爆炸。这使得特定领域的定制成为一项极具挑战性的任务。先前的工作探索使用强化学习(RL)和其他优化方法来自动探索大的设计空间。然而,这些方法传统上依赖于单一药剂RL/ML制剂。目前还不清楚如何可扩展的单剂配方,因为我们增加了设计空间的复杂性(例如,全栈片上系统设计)。因此,我们提出了一种替代配方,利用多Agent RL(MARL)来解决这个问题。使用MARL背后的关键思想是观察到不同子系统的参数或多或少是独立的,因此允许为每个代理分配分散的角色。我们测试这个假设,通过设计特定于域的DRAM内存控制器的几个工作负载的痕迹。我们的评估表明,MARL公式始终优于单代理RL基线,如近端策略优化和软演员批评在不同的目标,如低功耗和延迟。为此,这项工作开辟了新的和有前途的研究在MARL解决方案的硬件架构搜索的途径。

英文摘要:

Microprocessor architects are increasingly resorting to domain-specific customization in the quest for high-performance and energy-efficiency. As the systems grow in complexity, fine-tuning architectural parameters across multiple sub-systems (e.g., datapath, memory blocks in different hierarchies, interconnects, compiler optimization, etc.) quickly results in a combinatorial explosion of design space. This makes domain-specific customization an extremely challenging task. Prior work explores using reinforcement learning (RL) and other optimization methods to automatically explore the large design space. However, these methods have traditionally relied on single-agent RL/ML formulations. It is unclear how scalable single-agent formulations are as we increase the complexity of the design space (e.g., full stack System-on-Chip design). Therefore, we propose an alternative formulation that leverages Multi-Agent RL (MARL) to tackle this problem. The key idea behind using MARL is an observation that parameters across different sub-systems are more or less independent, thus allowing a decentralized role assigned to each agent. We test this hypothesis by designing domain-specific DRAM memory controller for several workload traces. Our evaluation shows that the MARL formulation consistently outperforms single-agent RL baselines such as Proximal Policy Optimization and Soft Actor-Critic over different target objectives such as low power and latency. To this end, this work opens the pathway for new and promising research in MARL solutions for hardware architecture search.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑