将存储端点集成到 Rucio 数据湖中:作为 SKA 区域中心网络原型的一项活动
Integration of storage endpoints into a Rucio data lake, as an activity to prototype a SKA Regional Centres Network
AI总结:
本文介绍 SRCNet 数据管理原型中,如何在 SKAO 的 Rucio 数据湖内集成基于 StoRM 和 WebDAV 的新 Rucio Storage Element,并给出西班牙 SRC 原型的部署步骤。
摘要:
平方公里阵列(Square Kilometre Array,SKA)基础设施将由两台射电望远镜组成,它们将成为地球上最灵敏的望远镜。SKA 社区将必须处理和管理近百亿亿级(near exascale)数据,这将是未来几年面临的一项技术挑战。在这方面,SKA 全球区域中心网络在数据分发和管理中发挥着关键作用。SRCNet 将提供分布式计算和数据存储容量,以及面向该网络的其他重要服务。在 SRCNet 内部,已经成立了若干团队,负责研究、设计和开发 5 个原型。其中一个原型与数据管理和分发有关,在该原型中已使用 Rucio 部署了一个数据湖。本文重点介绍若干团队为在 SKAO 数据湖内部署新的存储端点所执行的任务。特别是,我们将描述在西班牙 SRC 原型中,为向 Rucio 数据湖提供基于 StoRM 和 WebDAV 的新 Rucio Storage Element 所需服务的步骤和部署说明。
英文摘要:
The Square Kilometre Array (SKA) infrastructure will consist of two radio telescopes that will be the most sensitive telescopes on Earth. The SKA community will have to process and manage near exascale data, which will be a technical challenge for the coming years. In this respect, the SKA Global Network of Regional Centres plays a key role in data distribution and management. The SRCNet will provide distributed computing and data storage capacity, as well as other important services for the network. Within the SRCNet, several teams have been set up for the research, design and development of 5 prototypes. One of these prototypes is related to data management and distribution, where a data lake has been deployed using Rucio. In this paper we focus on the tasks performed by several of the teams to deploy new storage endpoints within the SKAO data lake. In particular, we will describe the steps and deployment instructions for the services required to provide the Rucio data lake with a new Rucio Storage Element based on StoRM and WebDAV within the Spanish SRC prototype.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。