Savu:一个基于Python的MPI框架,用于多个N维大型断层扫描数据集的同时处理
Savu: A Python-based, MPI Framework for Simultaneous Processing of Multiple, N-dimensional, Large Tomography Datasets
摘要:
钻石光源(DLS),英国同步加速器设施,吸引了来自世界各地的科学家进行突破性的X射线实验。每年有超过3000名科学用户,通过实验光束线收集大量数据,其中在断层成像实验期间收集的数据量最大。断层扫描作为一种成像技术,越来越多的兴趣,导致扩大的范围内进行的实验,除了在每个实验的数据大小的增长。Savu是一个便携、灵活的科学处理管道,能够在PC上串行处理多个n维数据集,或在集群中并行处理。在DLS开发,并成功部署在光束线上,它使用模块化插件格式来实现实验特定的处理,并利用并行HDF5来消除RAM限制。Savu的设计,在本文中描述,重点是现有的和新的功能,灵活性和易用性,为用户和开发人员一样容易集成。
英文摘要:
Diamond Light Source (DLS), the UK synchrotron facility, attracts scientists from across the world to perform ground-breaking x-ray experiments. With over 3000 scientific users per year, vast amounts of data are collected across the experimental beamlines, with the highest volume of data collected during tomographic imaging experiments. A growing interest in tomography as an imaging technique, has led to an expansion in the range of experiments performed, in addition to a growth in the size of the data per experiment. Savu is a portable, flexible, scientific processing pipeline capable of processing multiple, n-dimensional datasets in serial on a PC, or in parallel across a cluster. Developed at DLS, and successfully deployed across the beamlines, it uses a modular plugin format to enable experiment-specific processing and utilises parallel HDF5 to remove RAM restrictions. The Savu design, described throughout this paper, focuses on easy integration of existing and new functionality, flexibility and ease of use for users and developers alike.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。