图像数据在LSST中的存储和检索
Abstracting the storage and retrieval of image data at the LSST
摘要:
编写通用数据处理管道要求算法代码不必知道文件的数据格式或这些文件的位置。在LSST,我们有一个被称为“数据管家”的软件系统,它从软件开发人员那里抽象出这些细节。科学家可以用他们理解的术语指定他们想要的数据集,例如过滤器,观察标识符,观察日期和仪器名称,Butler将其转换为一个或多个文件,这些文件作为单个Python对象读取并返回给他们。相反,一旦他们创建了一个新的数据集,他们就可以将它返回给管家,并带有一个描述其新状态的标签,管家可以以任何配置使用的格式编写它。所有配置都是YAML格式,支持标准默认值,同时允许重写。
英文摘要:
Writing generic data processing pipelines requires that the algorithmic code does not ever have to know about data formats of files, or the locations of those files. At LSST we have a software system known as "the Data Butler," that abstracts these details from the software developer. Scientists can specify the dataset they want in terms they understand, such as filter, observation identifier, date of observation, and instrument name, and the Butler translates that to one or more files which are read and returned to them as a single Python object. Conversely, once they have created a new dataset they can give it back to the Butler, with a label describing its new status, and the Butler can write it in whatever format it has been configured to use. All configuration is in YAML and supports standard defaults whilst allowing overrides.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。