凯克天文台档案馆（KOA）的实时数据摄取
Real-time Data Ingestion at the Keck Observatory Archive (KOA)
摘要：
自今年2月以来，KOA开始准备、传输和摄取近实时获取的数据;在大多数情况下，观测者在获取数据后一分钟内就可以通过KOA获得数据。到2022年夏季结束时，所有有源仪器的实时摄取将完成。该天文台正在支持现代Python数据简化管道的开发，该管道在交付时将在每晚结束时自动创建科学就绪的数据集，以便将其纳入存档。本演示将描述为支持实时数据摄取而开发的基础设施，这本身就是天文台实现端到端操作现代化的更大举措的一部分。在望远镜运行期间，当通过监测基于关键字的观测站控制系统识别新获取的文件时，WMKO的软件自动执行;凯克使用该系统执行几乎所有观测站功能。监控器使用控制系统中内置的回调函数开始文件的数据准备，以便在单独的基础上传输到存档：不需要调度脚本或文件系统相关的触发器。从Flask微框架调用的基于HTTP的系统使WMKO和NExScI之间的文件传输成为可能，并触发NExScI的数据摄取。NEXScI的摄取系统是一个紧凑（4 KB），高度容错，基于Python的系统。它使用共享文件系统将数据从WMKO传输到NExScI。摄取代码与仪器无关，仪器参数从配置文件读取。它取代了自2004年以来一直使用的笨重的（50 KB）基于C的系统。
英文摘要：
Since February of this year, KOA began to prepare, transfer, and ingest data as they were acquired in near-real time; in most cases data are available to observers through KOA within one minute of acquisition. Real-time ingestion will be complete for all active instruments by the end of Summer 2022. The observatory is supporting the development of modern Python data reduction pipelines, which when delivered, will automatically create science-ready data sets at the end of each night for ingestion into the archive. This presentation will describe the infrastructure developed to support real-time data ingestion, itself part of a larger initiative at the Observatory to modernize end-to-end operations. During telescope operations, the software at WMKO is executed automatically when a newly acquired file is recognized through monitoring a keyword-based observatory control system; this system is used at Keck to execute virtually all observatory functions. The monitor uses callbacks built into the control system to begin data preparation of files for transmission to the archive on an individual basis: scheduling scripts or file system related triggers are unnecessary. An HTTP-based system called from the Flask micro-framework enables file transfers between WMKO and NExScI and triggers data ingestion at NExScI. The ingestion system at NEXScI is a compact (4 KLOC), highly fault-tolerant, Python-based system. It uses a shared file system to transfer data from WMKO to NExScI. The ingestion code is instrument agnostic, with instrument parameters read from configuration files. It replaces an unwieldy (50 KLOC) C-based system that had been in use since 2004.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
