基于Dyna风格强化学习的模拟电路设计
Analog Circuit Design with Dyna-Style Reinforcement Learning
摘要：
在这项工作中，我们提出了一种基于学习的模拟电路设计方法，其目标是在一定的设计约束条件下优化电路性能。使这个问题具有挑战性的优化的方面之一是，用仿真来测量候选配置的性能可能在计算上是昂贵的，特别是在布局后设计中。此外，大量的设计约束和相关量之间的相互作用使得问题复杂。因此，为了更好地促进支持人类设计师，期望获得关于可行解的整个空间的知识。为了应对这些挑战，我们从基于模型的强化学习中获得灵感，并提出了一种具有两个关键属性的方法。首先，它学习奖励模型，即，代理模型的性能由神经网络近似，减少了所需的仿真次数。其次，它使用一个随机策略生成器来探索满足约束的多样解空间。我们将这些结合在Dyna风格的优化框架中，我们称之为DynaOpt，并根据经验评估两级运算放大器的电路基准的性能。结果表明，与使用无模型方法进行20，000次电路仿真来训练策略相比，DynaOpt仅用500次仿真就从头开始学习，从而实现了更好的性能。
英文摘要：
In this work, we present a learning based approach to analog circuit design, where the goal is to optimize circuit performance subject to certain design constraints. One of the aspects that makes this problem challenging to optimize, is that measuring the performance of candidate configurations with simulation can be computationally expensive, particularly in the post-layout design. Additionally, the large number of design constraints and the interaction between the relevant quantities makes the problem complex. Therefore, to better facilitate supporting the human designers, it is desirable to gain knowledge about the whole space of feasible solutions. In order to tackle these challenges, we take inspiration from model-based reinforcement learning and propose a method with two key properties. First, it learns a reward model, i.e., surrogate model of the performance approximated by neural networks, to reduce the required number of simulation. Second, it uses a stochastic policy generator to explore the diverse solution space satisfying constraints. Together we combine these in a Dyna-style optimization framework, which we call DynaOpt, and empirically evaluate the performance on a circuit benchmark of a two-stage operational amplifier. The results show that, compared to the model-free method applied with 20,000 circuit simulations to train the policy, DynaOpt achieves even much better performance by learning from scratch with only 500 simulations.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
