雄心勃勃的数据科学可以无痛
Ambitious Data Science Can Be Painless
摘要:
现代数据科学研究可能涉及大量的计算实验;一个雄心勃勃的计算领域博士可能会做消耗数百万CPU小时的实验。传统的计算实践,即研究人员使用笔记本电脑或共享校园居民资源,不足以进行我们现在在数据科学中看到的大规模和不同范围的实验。另一方面,现代云计算似乎承诺了无限的计算资源,可以自定义配置,似乎为雄心勃勃的数据驱动科学提供了一个强大的新场所。充分利用云计算,可以在固定时间内完成的工作量可以增加几个数量级。尽管基于云的实验在抽象上可能很强大,但它尚未成为许多学科研究人员的标准选择。在当今的云系统中实际进行大规模计算实验的前景使潜在用户面临着艰巨的挑战。主要考虑因素包括:(i)今天的云计算界面看起来很复杂,(ii)难以执行大量作业,以及(iii)难以监控和组合大量单独结果。因此,"赤手空拳"开始一项大规模实验似乎问题很大,而且容易迅速"使研究人员精疲力竭"。新的软件栈正在出现,使大规模的云实验相对无痛。这些堆栈通过系统化实验定义、自动化任务分配和管理以及允许轻松获取结果和文档来简化实验。在这篇文章中,我们讨论了几个无痛计算堆栈,这些堆栈抽象了大规模实验的困难,从而允许为科学发现进行雄心勃勃的实验。
英文摘要:
Modern data science research can involve massive computational experimentation; an ambitious PhD in computational fields may do experiments consuming several million CPU hours. Traditional computing practices, in which researchers use laptops or shared campus-resident resources, are inadequate for experiments at the massive scale and varied scope that we now see in data science. On the other hand, modern cloud computing promises seemingly unlimited computational resources that can be custom configured, and seems to offer a powerful new venue for ambitious data-driven science. Exploiting the cloud fully, the amount of work that could be completed in a fixed amount of time can expand by several orders of magnitude. As potentially powerful as cloud-based experimentation may be in the abstract, it has not yet become a standard option for researchers in many academic disciplines. The prospect of actually conducting massive computational experiments in today's cloud systems confronts the potential user with daunting challenges. Leading considerations include: (i) the seeming complexity of today's cloud computing interface, (ii) the difficulty of executing an overwhelmingly large number of jobs, and (iii) the difficulty of monitoring and combining a massive collection of separate results. Starting a massive experiment `bare-handed' seems therefore highly problematic and prone to rapid `researcher burn out'. New software stacks are emerging that render massive cloud experiments relatively painless. Such stacks simplify experimentation by systematizing experiment definition, automating distribution and management of tasks, and allowing easy harvesting of results and documentation. In this article, we discuss several painless computing stacks that abstract away the difficulties of massive experimentation, thereby allowing a proliferation of ambitious experiments for scientific discovery.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。