arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1809.07693cs.DCcs.SE
收藏

无服务器平台无关计算实验管理工具

A Serverless Tool for Platform Agnostic Computational Experiment Management

Gregory Kiar, Shawn T Brown, Tristan Glatard, Alan C Evans

摘要:

在数据汇集和越来越多的计算密集型工具的支持下,神经科学已经进入大数据和高性能计算(HPC)领域。虽然管理HPC实验需要相当的技术敏锐度,但已经开发了平台和标准来减轻科学家的这一负担。虽然门户网站使资源可以广泛访问,但数据组织(如Brain Imaging数据结构)和工具描述语言(如Boutiques)为研究人员提供了一个立足点,可以使用他们自己的数据集、管道和环境来解决这些问题。虽然这些标准降低了神经科学应用采用HPC和云系统的障碍,但它们仍然需要整合不同的特定领域知识。我们介绍了Clowdr,一个轻量级的工具,用于在HPC系统和云上启动实验,记录丰富的执行记录,并实现实验摘要和结果的可访问共享。Clowdr独特地位于网络平台和用于实验管理的裸机应用程序之间,通过保留自己动手解决方案的灵活性,同时为开发,部署和传播神经科学分析提供低障碍。

英文摘要:

Neuroscience has been carried into the domain of big data and high performance computing (HPC) on the backs of initiatives in data collection and an increasingly compute-intensive tools. While managing HPC experiments requires considerable technical acumen, platforms and standards have been developed to ease this burden on scientists. While web-portals make resources widely accessible, data organizations such as the Brain Imaging Data Structure and tool description languages such as Boutiques provide researchers with a foothold to tackle these problems using their own datasets, pipelines, and environments. While these standards lower the barrier to adoption of HPC and cloud systems for neuroscience applications, they still require the consolidation of disparate domain-specific knowledge. We present Clowdr, a lightweight tool to launch experiments on HPC systems and clouds, record rich execution records, and enable the accessible sharing of experimental summaries and results. Clowdr uniquely sits between web platforms and bare-metal applications for experiment management by preserving the flexibility of do-it-yourself solutions while providing a low barrier for developing, deploying and disseminating neuroscientific analysis.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑