TrimTuner:通过子采样有效优化云中的机器学习作业
TrimTuner: Efficient Optimization of Machine Learning Jobs in the Cloud via Sub-Sampling
摘要:
这项工作介绍了TrimTuner,这是第一个在云中优化机器学习作业的系统,它利用子采样技术来降低优化过程的成本,同时考虑到用户指定的约束。TrimTuner联合优化云和应用程序特定的参数,与云优化的最新技术不同,它避免了每次对新配置进行采样时使用完整训练集对模型进行训练的需要。事实上,通过利用子采样技术和比原始数据小60倍的数据集,我们证明TrimTuner可以将优化过程的成本降低50倍。此外,TrimTuner相对于使用子采样技术的超参数优化技术,将推荐过程加快了65倍。这种改进的原因是双重的:i)新颖的域特定启发式,其减少了必须评估获取函数的配置的数量; ii)采用决策树的集合,其使得能够将推荐过程的速度提高一个额外的数量级。
英文摘要:
This work introduces TrimTuner, the first system for optimizing machine learning jobs in the cloud to exploit sub-sampling techniques to reduce the cost of the optimization process while keeping into account user-specified constraints. TrimTuner jointly optimizes the cloud and application-specific parameters and, unlike state of the art works for cloud optimization, eschews the need to train the model with the full training set every time a new configuration is sampled. Indeed, by leveraging sub-sampling techniques and data-sets that are up to 60x smaller than the original one, we show that TrimTuner can reduce the cost of the optimization process by up to 50x. Further, TrimTuner speeds-up the recommendation process by 65x with respect to state of the art techniques for hyper-parameter optimization that use sub-sampling techniques. The reasons for this improvement are twofold: i) a novel domain specific heuristic that reduces the number of configurations for which the acquisition function has to be evaluated; ii) the adoption of an ensemble of decision trees that enables boosting the speed of the recommendation process by one additional order of magnitude.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。