灯塔强化学习：通过策略性重置点实现样本高效的电路优化

中文摘要 AI 辅助

本文介绍了灯塔强化学习（Lighthouse RL），一种用于模拟电路规模确定的样本高效强化学习方法。传统方法在不同性能目标间缺乏通用性，标准强化学习方法会在无前景区域浪费资源。我们的方法通过策略性重置策略解决这些低效问题，该策略从训练中发现的高性能配置（即“灯塔”）初始化情节。这些更接近目标的状态引导探索向有前景区域。与文献中的强化学习和贝叶斯优化方法相比，在二维基准问题和两个模拟电路上展示了方法的有效性，在样本效率、优化性能、通用性和目标最大化方面有显著提升。这种效率对计算昂贵的黑箱优化问题尤其有价值，重置策略可作为基于强化学习的优化方法的即插即用增强。

英文摘要

In this paper, we introduce Lighthouse RL, a sample-efficient reinforcement learning (RL) approach for analog circuit sizing. Traditional methods lack generalization across different performance targets, while standard RL approaches waste resources exploring unpromising regions. Our method addresses these inefficiencies through a strategic reset strategy that initializes episodes from high-performing configurations discovered during training, called "lighthouses". These states, which are closer to the target objectives, guide exploration toward promising regions. When compared to RL and Bayesian optimization methods from the literature, we demonstrate the effectiveness of our approach on a 2D benchmark problem and on two analog circuits, showing significant improvements in sample efficiency (up to 1.72x faster), optimization performance (100% vs. 0-87% success rate), generalization (75% vs. 0-50% extrapolation success), and objective maximization. This efficiency is particularly valuable for computationally expensive black-box optimization problems, and our reset strategy can be used as a plug-and-play enhancement for any RL-based optimization approach.

发表机构

• Sony Group Corporation（索尼集团公司）
• EPFL（洛桑联邦理工学院）
• Sony Semiconductor Solutions（索尼半导体解决方案公司）
• TU Munich（慕尼黑工业大学）

机构由 AI 辅助整理，请以论文原文为准。