使用 openPMD 和 Darshan I/O 监测在 Particle-in-Cell 蒙特卡洛模拟中实现高吞吐量并行 I/O
Enabling High-Throughput Parallel I/O in Particle-in-Cell Monte Carlo Simulations with openPMD and Darshan I/O Monitoring
AI总结：
本文针对聚变装置等离子体模拟中缺乏并行I/O导致的性能瓶颈，在BIT1代码中引入openPMD接口并结合ADIOS2、数据压缩与Lustre条带化等优化技术，显著提升了写入吞吐量与存储效率，成功实现高吞吐量并行I/O。
摘要：
聚变装置中等离子体动力学的大规模 HPC 模拟需要高效的并行 I/O，以避免减慢模拟速度并实现对关键信息的后处理。缺乏并行 I/O 能力的此类复杂模拟可能会遇到性能瓶颈，阻碍其在数据密集型计算任务中的有效性。在这项工作中，我们专注于在 Particle-in-Cell 蒙特卡洛模拟中引入并提升并行 I/O 操作的效率。我们首先评估了 BIT1（一种大规模并行静电 PIC MC 代码）的可扩展性，使用 HPC I/O 性能监测工具 Darshan 确定了其初始写入吞吐量能力和性能瓶颈。我们设计并开发了一个 openPMD I/O 接口的适配器，允许我们使用 BP4 后端将 PIC 粒子和场信息流式传输到 I/O，该后端针对 I/O 效率进行了深度优化，包括高效的 ADIOS2 接口。接下来，我们探索了数据压缩、聚合和 Lustre 文件条带化等高级优化技术，在提升数据存储效率的同时实现了写入吞吐量的提升。最后，我们分析了通过将 openPMD 与 BP4 格式的快速元数据提取相集成所实现的增强型高吞吐量并行 I/O 和存储能力。我们的研究表明，openPMD 与高级 I/O 优化的集成显著增强了 BIT1 的 I/O 性能和存储能力，成功引入了高吞吐量并行 I/O 并超越了传统文件 I/O 的能力。
英文摘要：
Large-scale HPC simulations of plasma dynamics in fusion devices require efficient parallel I/O to avoid slowing down the simulation and to enable the post-processing of critical information. Such complex simulations lacking parallel I/O capabilities may encounter performance bottlenecks, hindering their effectiveness in data-intensive computing tasks. In this work, we focus on introducing and enhancing the efficiency of parallel I/O operations in Particle-in-Cell Monte Carlo simulations. We first evaluate the scalability of BIT1, a massively-parallel electrostatic PIC MC code, determining its initial write throughput capabilities and performance bottlenecks using an HPC I/O performance monitoring tool, Darshan. We design and develop an adaptor to the openPMD I/O interface that allows us to stream PIC particle and field information to I/O using the BP4 backend, aggressively optimized for I/O efficiency, including the highly efficient ADIOS2 interface. Next, we explore advanced optimization techniques such as data compression, aggregation, and Lustre file striping, achieving write throughput improvements while enhancing data storage efficiency. Finally, we analyze the enhanced high-throughput parallel I/O and storage capabilities achieved through the integration of openPMD with rapid metadata extraction in BP4 format. Our study demonstrates that the integration of openPMD and advanced I/O optimizations significantly enhances BIT1's I/O performance and storage capabilities, successfully introducing high throughput parallel I/O and surpassing the capabilities of traditional file I/O.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
