ucTrace:一种用于UCX驱动通信的多层剖析工具
ucTrace: A Multi-Layer Profiling Tool for UCX-driven Communication
AI总结:
ucTrace是一种用于HPC环境的多层剖析工具,通过在UCX层面剖析消息传递,帮助优化和调试大规模工作负载的通信性能。
摘要:
UCX是一种用于HPC系统中低延迟、高带宽通信的通信框架。凭借其统一的API,UCX在多节点CPU-GPU集群中实现了高效的跨节点数据传输。UCX广泛用作MPI的传输层,特别是在GPU-aware实现中。然而,现有的剖析工具缺乏在UCX层面的细粒度通信轨迹,无法捕获传输层行为,或仅限于特定的MPI实现。为解决这些差距,我们引入了ucTrace,一种新型的剖析工具,它暴露并可视化HPC环境中的UCX驱动通信。ucTrace通过在UCX层面剖析消息传递,将主机和设备(如GPU和NIC)之间的操作直接链接到其起源的MPI函数。通过交互式可视化进程和设备特定的交互,ucTrace帮助系统管理员、库和应用程序开发者优化性能并调试大规模工作负载中的通信模式。我们通过一系列实验展示了ucTrace的功能,包括不同UCX设置下的MPI点对点行为、不同MPI库之间的Allreduce比较、线性求解器的通信分析、NUMA绑定效果以及带有GPU加速的大规模GROMACS MD模拟的剖析。ucTrace可在https://github.com/ParCoreLab/ucTrace上公开获取。
英文摘要:
UCX is a communication framework that enables low-latency, high-bandwidth communication in HPC systems. With its unified API, UCX facilitates efficient data transfers across multi-node CPU-GPU clusters. UCX is widely used as the transport layer for MPI, particularly in GPU-aware implementations. However, existing profiling tools lack fine-grained communication traces at the UCX level, do not capture transport-layer behavior, or are limited to specific MPI implementations. To address these gaps, we introduce ucTrace, a novel profiler that exposes and visualizes UCX-driven communication in HPC environments. ucTrace provides insights into MPI workflows by profiling message passing at the UCX level, linking operations between hosts and devices (e.g., GPUs and NICs) directly to their originating MPI functions. Through interactive visualizations of process- and device-specific interactions, ucTrace helps system administrators, library and application developers optimize performance and debug communication patterns in large-scale workloads. We demonstrate ucTrace's features through a wide range of experiments including MPI point-to-point behavior under different UCX settings, Allreduce comparisons across MPI libraries, communication analysis of a linear solver, NUMA binding effects, and profiling of GROMACS MD simulations with GPU acceleration at scale. ucTrace is publicly available at https://github.com/ParCoreLab/ucTrace.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。