CHASE-CI中工作流驱动的分布式机器学习：认知硬件和软件生态系统社区基础设施
Workflow-Driven Distributed Machine Learning in CHASE-CI: A Cognitive Hardware and Software Ecosystem Community Infrastructure
摘要：
过去二十年来，数据、计算和网络方面的进步导致许多应用领域发生了转变，其中包括作为科学过程一部分的大数据机器学习，这需要集成和分布式硬件和软件基础设施的新功能。本文贡献了一种工作流驱动的方法，动态数据驱动的应用程序开发的顶部一种新的网络网络基础设施称为CHASE-CI。我们特别介绍：1）CHASE-CI的架构，CHASE-CI是一个分布式快速GPU设备网络，用于机器学习和通过Kubernetes在高速（10- 100 Gbps）太平洋研究平台（PRP）; 2）将这样的网络转变成用于大数据分析的分布式计算机所需的机器学习软件容器化方法和库; 3）大气科学案例研究，只能通过CHASE-CI这样的基础设施进行扩展; 4）虚拟集群管理的能力，用于动态扩展方式的数据通信和分析，以及近实时地在专用可视化设施中跨网络进行可视化;以及5）逐步的工作流和性能测量方法，其使得能够利用CHASE-CI网络和容器管理基础设施的动态体系结构。
英文摘要：
The advances in data, computing and networking over the last two decades led to a shift in many application domains that includes machine learning on big data as a part of the scientific process, requiring new capabilities for integrated and distributed hardware and software infrastructure. This paper contributes a workflow-driven approach for dynamic data-driven application development on top of a new kind of networked Cyberinfrastructure called CHASE-CI. In particular, we present: 1) The architecture for CHASE-CI, a network of distributed fast GPU appliances for machine learning and storage managed through Kubernetes on the high-speed (10-100Gbps) Pacific Research Platform (PRP); 2) A machine learning software containerization approach and libraries required for turning such a network into a distributed computer for big data analysis; 3) An atmospheric science case study that can only be made scalable with an infrastructure like CHASE-CI; 4) Capabilities for virtual cluster management for data communication and analysis in a dynamically scalable fashion, and visualization across the network in specialized visualization facilities in near real-time; and, 5) A step-by-step workflow and performance measurement approach that enables taking advantage of the dynamic architecture of the CHASE-CI network and container management infrastructure.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
