Apollo：可迁移的架构探索
Apollo: Transferable Architecture Exploration
摘要：
摩尔定律的即将终结和深度学习的日益普及推动了针对特定神经架构进行优化的定制加速器的设计。这种加速器的架构探索形成了一个具有挑战性的约束优化问题，在一个复杂的，高维的，结构化的输入空间与昂贵的评估目标函数。用于加速器设计的现有方法是样本效率低的，并且不能在具有不同设计约束（诸如面积和/或延迟预算或神经架构配置）的相关优化任务之间传递知识。在这项工作中，我们提出了一个可转移的架构探索框架，称为阿波罗，利用黑盒功能优化的最新进展，为样本高效的加速器设计。我们使用这个框架来优化具有替代设计约束的不同神经架构的加速器配置。我们表明，我们的框架发现高回报设计配置（高达24.6%的加速比）更采样效率比基线黑盒优化方法。我们进一步表明，通过在具有不同设计约束的目标架构之间转移知识，Apollo能够更快地找到最佳配置，并且通常具有更好的目标值（最高可提高25%）。这一令人鼓舞的结果描绘了一条有希望的前进道路，以促进产生更高质量的加速器。
英文摘要：
The looming end of Moore's Law and ascending use of deep learning drives the design of custom accelerators that are optimized for specific neural architectures. Architecture exploration for such accelerators forms a challenging constrained optimization problem over a complex, high-dimensional, and structured input space with a costly to evaluate objective function. Existing approaches for accelerator design are sample-inefficient and do not transfer knowledge between related optimizations tasks with different design constraints, such as area and/or latency budget, or neural architecture configurations. In this work, we propose a transferable architecture exploration framework, dubbed Apollo, that leverages recent advances in black-box function optimization for sample-efficient accelerator design. We use this framework to optimize accelerator configurations of a diverse set of neural architectures with alternative design constraints. We show that our framework finds high reward design configurations (up to 24.6% speedup) more sample-efficiently than a baseline black-box optimization approach. We further show that by transferring knowledge between target architectures with different design constraints, Apollo is able to find optimal configurations faster and often with better objective value (up to 25% improvements). This encouraging outcome portrays a promising path forward to facilitate generating higher quality accelerators.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
