arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1503.08482cs.DB
收藏

论亿级科学元数据管理

Towards Exascale Scientific Metadata Management

Spyros Blanas, Surendra Byna

摘要:

技术和计算硬件的进步使所有科学领域的科学家能够使用大规模模拟或观测设施产生大量数据。在这个数据泛滥的时代,数据生产和分析阶段之间的有效协调取决于描述科学数据集的元数据的可用性。现有的工作流引擎已经捕获了有限形式的元数据,以提供有关数据的身份和血统的起源信息。然而,许多由模拟、实验和分析产生的数据仍然需要由领域科学家以特定的方式手动注释。系统和透明地获取丰富的元数据成为维持和加快科学创新步伐的重要先决条件。然而,无处不在的和域不可知的元数据管理基础设施,可以满足极端规模的科学的需求是值得注意的是,它的缺乏。为了解决科学数据管理研究和实践中的这一差距,我们提出了一种集成方法,(1)自动捕获和操作信息丰富的元数据,而数据正在产生或分析,(2)存储元数据在每个数据集渗透元数据遗忘的过程中,并通过建立和标准化的数据访问接口查询元数据。我们激励需要使用等离子体物理,气候建模和神经科学的应用程序,提出的综合方法,然后讨论研究的挑战和可能的解决方案。

英文摘要:

Advances in technology and computing hardware are enabling scientists from all areas of science to produce massive amounts of data using large-scale simulations or observational facilities. In this era of data deluge, effective coordination between the data production and the analysis phases hinges on the availability of metadata that describe the scientific datasets. Existing workflow engines have been capturing a limited form of metadata to provide provenance information about the identity and lineage of the data. However, much of the data produced by simulations, experiments, and analyses still need to be annotated manually in an ad hoc manner by domain scientists. Systematic and transparent acquisition of rich metadata becomes a crucial prerequisite to sustain and accelerate the pace of scientific innovation. Yet, ubiquitous and domain-agnostic metadata management infrastructure that can meet the demands of extreme-scale science is notable by its absence. To address this gap in scientific data management research and practice, we present our vision for an integrated approach that (1) automatically captures and manipulates information-rich metadata while the data is being produced or analyzed and (2) stores metadata within each dataset to permeate metadata-oblivious processes and to query metadata through established and standardized data access interfaces. We motivate the need for the proposed integrated approach using applications from plasma physics, climate modeling and neuroscience, and then discuss research challenges and possible solutions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑