arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1811.04740cs.DC
收藏

数据托盘:集装箱化存储,可重复性和可追溯性

Data Pallets: Containerizing Storage For Reproducibility and Traceability

Jay Lofstead, Joshua Baker, Andrew Younge

摘要:

信任模拟输出对于桑迪亚的任务目标至关重要。鉴于国家条约义务,我们依靠这些模拟来执行我们的高后果任务。其他科学和建模应用程序虽然可能具有高后果的结果,但仍然需要最高级别的信任,以便将结果用作实际应用和未来研究的基础。为此,计算社区开发了工作流和出处系统,以帮助自动化模拟和建模执行,以及准确确定如何创建某些输出,以便从数据中得出结论。目前的工作流和起源系统的方法都是在用户级别,几乎没有系统级别的支持,使他们脆弱,难以使用,和不完整的解决方案。容器技术的引入是封装和跟踪用于创建数据和产生见解的工件的第一步,但其当前的实现仅专注于使其易于在隔离的“沙箱”中部署应用程序,并保持严格的只读模式以避免对应用程序的任何潜在更改。所有存储活动仍在使用系统级共享存储。这个项目探索扩展容器的概念,将存储作为一种新的容器类型,我们称之为\emph{data pallets}。数据托盘可能是可写的,由系统基于IO活动自动生成,并可用作将包含的数据链接回用于创建数据的应用程序和输入平台的一种方式。

英文摘要:

Trusting simulation output is crucial for Sandia's mission objectives. We rely on these simulations to perform our high-consequence mission tasks given national treaty obligations. Other science and modeling applications, while they may have high-consequence results, still require the strongest levels of trust to enable using the result as the foundation for both practical applications and future research. To this end, the computing community has developed workflow and provenance systems to aid in both automating simulation and modeling execution as well as determining exactly how was some output was created so that conclusions can be drawn from the data. Current approaches for workflows and provenance systems are all at the user level and have little to no system level support making them fragile, difficult to use, and incomplete solutions. The introduction of container technology is a first step towards encapsulating and tracking artifacts used in creating data and resulting insights, but their current implementation is focused solely on making it easy to deploy an application in an isolated "sandbox" and maintaining a strictly read-only mode to avoid any potential changes to the application. All storage activities are still using the system-level shared storage. This project explores extending the container concept to include storage as a new container type we call \emph{data pallets}. Data Pallets are potentially writeable, auto generated by the system based on IO activities, and usable as a way to link the contained data back to the application and input deck used to create it.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑