arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.05244cs.DC
收藏

CRIU -- 面向计算模拟与科学应用的用户空间检查点恢复

CRIU -- Checkpoint Restore in Userspace for computational simulations and scientific applications

Fabio Andrijauskas, Igor Sfiligoi, Diego Davila, Aashay Arora, Jonathan Guiang, Brian Bockelman, Greg Thain, Frank Wurthwein

AI总结:

本文评估了在OSG的OSPool HTCondor环境中使用开源工具CRIU对长时间运行的科学计算作业进行检查点恢复的可行性,发现其功能适用于多数场景,但存在限制。

摘要:

创造新材料、发现新药物以及模拟系统是研究与创新的关键过程,需要大量的计算能力。虽然许多应用可以被拆分为大量较小的独立任务,但有些应用无法拆分,可能需要数小时或数周才能运行完成。为了更好地管理这些长时间运行的作业,理想的做法是在任意时间点停止它们,并在之后于另一计算资源上继续其计算;这通常被称为检查点技术。虽然某些应用可以通过编程方式管理检查点,但更可取的是批处理调度系统能够独立完成此操作。本文评估了使用CRIU(用户空间检查点恢复)的可行性,这是一款面向GNU/Linux环境的开源工具,重点针对OSG的OSPool HTCondor设置。CRIU允许将进程状态检查点到磁盘映像中,并能无缝处理打开的文件和已建立的网络连接。此外,它还可以对传统Linux进程和容器化工作负载进行检查点操作。该功能似乎足以满足OSPool中支持的许多场景。然而,一些限制使其无法在所有情况下使用。

英文摘要:

Creating new materials, discovering new drugs, and simulating systems are essential processes for research and innovation and require substantial computational power. While many applications can be split into many smaller independent tasks, some cannot and may take hours or weeks to run to completion. To better manage those longer-running jobs, it would be desirable to stop them at any arbitrary point in time and later continue their computation on another compute resource; this is usually referred to as checkpointing. While some applications can manage checkpointing programmatically, it would be preferable if the batch scheduling system could do that independently. This paper evaluates the feasibility of using CRIU (Checkpoint Restore in Userspace), an open-source tool for the GNU/Linux environments, emphasizing the OSG's OSPool HTCondor setup. CRIU allows checkpointing the process state into a disk image and can deal with both open files and established network connections seamlessly. Furthermore, it can checkpoint traditional Linux processes and containerized workloads. The functionality seems adequate for many scenarios supported in the OSPool. However, some limitations prevent it from being usable in all circumstances.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑