arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2506.03474cs.LGcs.AIcs.AR
收藏

CORE:面向仿真引导神经网络加速器设计的约束感知一步强化学习

CORE: Constraint-Aware One-Step Reinforcement Learning for Simulation-Guided Neural Network Accelerator Design

University of California, Berkeley(加州大学伯克利分校);Futurewei Technologies(未来科技)

Yifeng Xiao, Yurong Xu, Ning Yan, Masood Mortazavi, Pierluigi Nuzzo

AI总结:

本文提出CORE,一种约束感知的一步强化学习方法,通过结构化分布采样、基于缩放图的解码器和无批评家的代理目标,在仿真引导的设计空间探索中高效平衡采样效率与约束满足,并在神经网络加速器协同设计中显著优于现有基线。

摘要:

基于仿真的设计空间探索(DSE)旨在复杂约束和高昂评估成本下高效优化高维结构化设计。现有方法,包括启发式方法和多步强化学习(RL)方法,由于稀疏、延迟的反馈以及庞大的混合动作空间,难以在采样效率和约束满足之间取得平衡。本文提出CORE,一种面向仿真引导DSE的约束感知一步强化学习方法。在CORE中,策略智能体通过在设计配置上定义结构化分布来学习采样设计配置,借助基于缩放图的解码器纳入依赖关系,并根据仿真获得的反馈通过奖励塑形来惩罚无效设计。CORE使用代理目标更新策略,该目标比较一个采样批次内设计的奖励,而无需学习价值函数。这种无批评家的形式通过鼓励选择更高奖励的设计实现高效学习。我们将CORE实例化用于神经网络加速器的硬件-映射协同设计,证明与最先进的基线相比,它显著提高了采样效率并获得了更好的加速器配置。我们的方法具有通用性,适用于广泛的离散-连续约束设计问题。

英文摘要:

Simulation-based design space exploration (DSE) aims to efficiently optimize high-dimensional structured designs under complex constraints and expensive evaluation costs. Existing approaches, including heuristic and multi-step reinforcement learning (RL) methods, struggle to balance sampling efficiency and constraint satisfaction due to sparse, delayed feedback, and large hybrid action spaces. In this paper, we introduce CORE, a constraint-aware, one-step RL method for simulationguided DSE. In CORE, the policy agent learns to sample design configurations by defining a structured distribution over them, incorporating dependencies via a scaling-graph-based decoder, and by reward shaping to penalize invalid designs based on the feedback obtained from simulation. CORE updates the policy using a surrogate objective that compares the rewards of designs within a sampled batch, without learning a value function. This critic-free formulation enables efficient learning by encouraging the selection of higher-reward designs. We instantiate CORE for hardware-mapping co-design of neural network accelerators, demonstrating that it significantly improves sample efficiency and achieves better accelerator configurations compared to state-of-the-art baselines. Our approach is general and applicable to a broad class of discrete-continuous constrained design problems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑