用于显微镜数据工作流程的处理和分析系统:Pycroscopy包生态系统
A processing and analytics system for microscopy data workflows: the Pycroscopy ecosystem of packages
摘要:
生物学和量子计算等不同领域的重大进展都依赖于大量的显微技术。所有光学、电子和扫描探针显微镜都随着新的检测器技术和光谱、成像和衍射的集成而进步。尽管这些工具大量增加,但在处理、分析、存储和检索所获得的数据集方面仍然存在重大瓶颈。除了缺乏文件标准之外,各个特定领域的分析包通常与底层数据集不相交。因此,跟踪分析和处理步骤对于最终用户来说仍然是乏味的,妨碍了再现性。在这里,我们介绍了包的pycroscopy生态系统,这是一个基于Python的开源生态系统,由一个通用的数据模型支撑。我们的数据模型,被称为N维光谱成像数据格式,实现在pycroscopy的duppy包。该软件包构建在dask阵列之上,因此利用了dask阵列属性,但扩展了它们,以加速显微镜相关的分析和可视化。显示了使用pycroscopy生态系统创建数据摄取和分析工作流的几个示例。采用这种标准化的程序将是至关重要的,以迎来下一代的自主仪器,其中处理,计算和元数据存储将是至关重要的整体实验操作。
英文摘要:
Major advancements in fields as diverse as biology and quantum computing have relied on a multitude of microscopic techniques. All optical, electron and scanning probe microscopy advanced with new detector technologies and integration of spectroscopy, imaging, and diffraction. Despite the considerable proliferation of these instruments, significant bottlenecks remain in terms of processing, analysis, storage, and retrieval of acquired datasets. Aside from the lack of file standards, individual domain-specific analysis packages are often disjoint from the underlying datasets. Thus, keeping track of analysis and processing steps remains tedious for the end-user, hampering reproducibility. Here, we introduce the pycroscopy ecosystem of packages, an open-source python-based ecosystem underpinned by a common data model. Our data model, termed the N-dimensional spectral imaging data format, is realized in pycroscopy's sidpy package. This package is built on top of dask arrays, thus leveraging dask array attributes but expanding them to accelerate microscopy-relevant analysis and visualization. Several examples of the use of the pycroscopy ecosystem to create workflows for data ingestion and analysis are shown. Adoption of such standardized routines will be critical to usher in the next generation of autonomous instruments where processing, computation, and meta-data storage will be critical to overall experimental operations.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。