arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2104.10746stat.MLcs.LG
收藏

受限时间约束下的自动模型训练

Automatic model training under restrictive time constraints

School of Mathematics, University of Leeds(利兹大学数学学院);The Alan Turing Institute(阿兰·图灵研究所)

Lukas Cironis, Jan Palczewski, Georgios Aivaliotis

AI总结:

提出AutoBCT算法,在受限时间下平衡模型质量与调优成本,通过马尔可夫决策过程在每个训练轮次决定终止或继续训练,并在随机森林和神经网络上验证性能。

摘要:

我们开发了一种超参数优化算法,名为自动预算受限训练(AutoBCT),该算法在模型质量与调优所需的计算成本之间取得平衡。超参数、模型质量与计算成本之间的关系必须被学习,并且这种学习被直接纳入优化问题中。在每个训练轮次,算法决定是终止训练还是继续训练,并且在继续训练的情况下,决定使用哪些超参数值。该决策在潜在的质量提升、额外的训练时间以及关于所学量的不确定性之间进行最优权衡。我们算法的性能在多个机器学习问题上得到了验证,这些问题涵盖随机森林和神经网络。我们的方法根植于具有部分信息的马尔可夫决策过程理论,并且我们开发了一种数值方法来计算价值函数和最优策略。

英文摘要:

We develop a hyperparameter optimisation algorithm, Automated Budget Constrained Training (AutoBCT), which balances the quality of a model with the computational cost required to tune it. The relationship between hyperparameters, model quality and computational cost must be learnt and this learning is incorporated directly into the optimisation problem. At each training epoch, the algorithm decides whether to terminate or continue training, and, in the latter case, what values of hyperparameters to use. This decision weighs optimally potential improvements in the quality with the additional training time and the uncertainty about the learnt quantities. The performance of our algorithm is verified on a number of machine learning problems encompassing random forests and neural networks. Our approach is rooted in the theory of Markov decision processes with partial information and we develop a numerical method to compute the value function and an optimal strategy.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑