arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1712.02446cs.LG
收藏

HyperPower:神经网络的功率和记忆约束超参数优化

HyperPower: Power- and Memory-Constrained Hyper-Parameter Optimization for Neural Networks

Carnegie Mellon University(卡内基梅隆大学);Google Research(谷歌研究院)

Dimitrios Stamoulis, Ermao Cai, Da-Cheng Juan, Diana Marculescu

摘要:

虽然选择神经网络(NN)的超参数迄今为止一直被视为一种艺术,但更复杂,更深入的架构的出现给设计师和机器学习(ML)从业者带来了越来越多的挑战,特别是当需要考虑功率和内存约束时。在这项工作中,我们提出了HyperPower,这是一个框架,可以在给定硬件平台上运行的NN的功率和内存约束超参数优化的背景下实现高效的贝叶斯优化和随机搜索。HyperPower是第一项工作,(i)表明功耗可以用作低成本,先验已知的约束,(ii)提出在GPU上执行的NN的功率和内存的预测模型。得益于HyperPower,通过约束不感知方法实现的函数求值数量和最佳测试误差分别快了112.99倍和30.12倍,同时从不考虑无效配置。HyperPower显著加快了超参数优化的速度,在给定的时间间隔内,与约束不感知方法相比,实现了高达57.20倍的函数计算,有效地提高了高达67.6%的准确性。

英文摘要:

While selecting the hyper-parameters of Neural Networks (NNs) has been so far treated as an art, the emergence of more complex, deeper architectures poses increasingly more challenges to designers and Machine Learning (ML) practitioners, especially when power and memory constraints need to be considered. In this work, we propose HyperPower, a framework that enables efficient Bayesian optimization and random search in the context of power- and memory-constrained hyper-parameter optimization for NNs running on a given hardware platform. HyperPower is the first work (i) to show that power consumption can be used as a low-cost, a priori known constraint, and (ii) to propose predictive models for the power and memory of NNs executing on GPUs. Thanks to HyperPower, the number of function evaluations and the best test error achieved by a constraint-unaware method are reached up to 112.99x and 30.12x faster, respectively, while never considering invalid configurations. HyperPower significantly speeds up the hyper-parameter optimization, achieving up to 57.20x more function evaluations compared to constraint-unaware methods for a given time interval, effectively yielding significant accuracy improvements by up to 67.6%.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑