未来十年的科学数据管理
Scientific Data Management in the Coming Decade
摘要:
这是一篇关于数据库和科学中心的数据密集型科学要求的思想文章。它认为,千万亿次规模的数据集将由科学中心提供,为通过智能笔记本访问数据的科学家提供大量的存储和处理。下一代科学仪器和模拟将生成这些千万亿次规模的数据集。由于需要公布和分享数据,而且需要通用分析和可视化工具,最终将在共同元数据标准上形成一种趋同。对数据库系统的评判标准将是它们是否支持这些元数据标准,以及它们是否有能力管理和访问千万亿次规模的数据集。以字节流文件为中心的程序化数据分析方法对于如此大的数据集来说既过于繁琐又过于串行。非过程化的查询和分析模式化的自描述数据既易于使用,又允许更多的并行性。
英文摘要:
This is a thought piece on data-intensive science requirements for databases and science centers. It argues that peta-scale datasets will be housed by science centers that provide substantial storage and processing for scientists who access the data via smart notebooks. Next-generation science instruments and simulations will generate these peta-scale datasets. The need to publish and share data and the need for generic analysis and visualization tools will finally create a convergence on common metadata standards. Database systems will be judged by their support of these metadata standards and by their ability to manage and access peta-scale datasets. The procedural stream-of-bytes-file-centric approach to data analysis is both too cumbersome and too serial for such large datasets. Non-procedural query and analysis of schematized self-describing data is both easier to use and allows much more parallelism.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。