arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv cs/0701037cs.DCcs.OS
收藏

DMTCP:用于集群计算和桌面的透明检查点

DMTCP: Transparent Checkpointing for Cluster Computations and the Desktop

Jason Ansel, Kapil Arya, Gene Cooperman

摘要:

DMTCP(Distributed MultiThreaded CheckPointing)是一个用于分布式应用程序的透明用户级检查点包。检查点设置和重新启动演示了20多个知名应用程序,包括MATLAB,Python,TightVNC,MPICH 2,OpenMPI和runCMS。RunCMS在内存中以680 MB的图像运行,其中包括540个动态库,并用于CERN大型强子对撞机的CMS实验。DMTCP透明地检查由许多节点、进程和线程组成的一般集群计算;以及典型的桌面应用程序。在128个分布式核心(32个节点)上,检查点和重启时间通常为2秒,运行时开销可以忽略不计。使用分叉检查点时,典型的检查点时间减少到0.2秒。实验结果表明,在中等规模的集群上,随着节点数量的增加,检查点时间几乎保持不变。DMTCP自动考虑fork、exec、ssh、互斥/信号量、TCP/IP套接字、UNIX域套接字、管道、ptys(伪终端)、终端模式、控制终端的所有权、信号处理程序、打开文件描述符、共享打开文件描述符、I/O(包括读取行库)、共享内存(通过mmap)、父子进程关系、pid虚拟化和其他操作系统工件。通过强调无特权的用户空间方法,从2.6.9到当前的2.6.28的Linux内核之间保持了兼容性。由于DMTCP是无特权的,并且不需要特殊的内核模块或内核补丁,因此DMTCP可以作为检查点重启模块在一些更大的包中合并和分发。

英文摘要:

DMTCP (Distributed MultiThreaded CheckPointing) is a transparent user-level checkpointing package for distributed applications. Checkpointing and restart is demonstrated for a wide range of over 20 well known applications, including MATLAB, Python, TightVNC, MPICH2, OpenMPI, and runCMS. RunCMS runs as a 680 MB image in memory that includes 540 dynamic libraries, and is used for the CMS experiment of the Large Hadron Collider at CERN. DMTCP transparently checkpoints general cluster computations consisting of many nodes, processes, and threads; as well as typical desktop applications. On 128 distributed cores (32 nodes), checkpoint and restart times are typically 2 seconds, with negligible run-time overhead. Typical checkpoint times are reduced to 0.2 seconds when using forked checkpointing. Experimental results show that checkpoint time remains nearly constant as the number of nodes increases on a medium-size cluster. DMTCP automatically accounts for fork, exec, ssh, mutexes/semaphores, TCP/IP sockets, UNIX domain sockets, pipes, ptys (pseudo-terminals), terminal modes, ownership of controlling terminals, signal handlers, open file descriptors, shared open file descriptors, I/O (including the readline library), shared memory (via mmap), parent-child process relationships, pid virtualization, and other operating system artifacts. By emphasizing an unprivileged, user-space approach, compatibility is maintained across Linux kernels from 2.6.9 through the current 2.6.28. Since DMTCP is unprivileged and does not require special kernel modules or kernel patches, DMTCP can be incorporated and distributed as a checkpoint-restart module within some larger package.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑