MLBench:机器学习云在结构化数据的二进制分类任务中有多好?
MLBench: How Good Are Machine Learning Clouds for Binary Classification Tasks on Structured Data?
摘要:
我们对主要云服务提供商提供的机器学习功能进行了实证研究,我们称之为机器学习云。机器学习云有望隐藏运行大规模机器学习的所有复杂性:用户只需指定要运行的机器学习任务,而不是指定如何运行机器学习任务,其余的由云计算。然而,提高抽象级别很少是免费的-性能损失是可能的。那么,当前的机器学习云在现实世界的机器学习工作负载上有多好?我们研究这个问题的重点是二进制分类问题。我们提出了mlbench,一个新的基准构建收获数据集从Kaggle比赛。然后,我们将Kaggle提供的最佳获奖代码的性能与在mlbench上运行Azure和Amazon的机器学习云的性能进行了比较。我们的比较研究揭示了现有机器学习云的优势和劣势,并指出了未来潜在的改进方向。
英文摘要:
We conduct an empirical study of machine learning functionalities provided by major cloud service providers, which we call machine learning clouds. Machine learning clouds hold the promise of hiding all the sophistication of running large-scale machine learning: Instead of specifying how to run a machine learning task, users only specify what machine learning task to run and the cloud figures out the rest. Raising the level of abstraction, however, rarely comes free - a performance penalty is possible. How good, then, are current machine learning clouds on real-world machine learning workloads? We study this question with a focus on binary classication problems. We present mlbench, a novel benchmark constructed by harvesting datasets from Kaggle competitions. We then compare the performance of the top winning code available from Kaggle with that of running machine learning clouds from both Azure and Amazon on mlbench. Our comparative study reveals the strength and weakness of existing machine learning clouds and points out potential future directions for improvement.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。