arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2407.18795cs.DC
收藏

并行计算讲义

Lectures on Parallel Computing

TU Wien(维也纳工业大学)

Jesper Larsson Träff

AI总结:

本并行计算讲义面向本科并行基础课与研究生高性能计算相关课程,讲解并行算法理论、OpenMP/MPI/pthreads编程框架与类C代码示例,配套习题可用于自学与课程项目,相关通用原则也适用于GPU架构。

摘要:

这些讲义旨在配套一门虚拟的本科阶段一至两学期的并行计算基础课程,同时也可作为高性能计算、并行算法与共享内存多处理器编程等研究生课程的背景资料与参考。讲义介绍了用于表述、分析和评判并行算法的理论概念与工具,并详细讲解了两种应用最广泛的具体框架——OpenMP和MPI,以及用于编写并行程序的线程接口pthreads,这些内容适用于共享内存或分布式内存并行计算机,重点讲解通用概念与原则。代码示例采用类C风格,其中许多是可实际运行的正确C代码。本讲义有意不涵盖GPU架构与GPU编程,但其中涉及的通用考量、指导方针和原则(时间、工作量、成本、效率、可扩展性、内存结构与带宽)对于高效利用各类GPU架构同样适用。同样地,讲义仅聚焦于确定性算法,不涉及随机化方法。学习本材料的学生如果花时间从视觉层面理解概念与算法,会发现这很有启发意义。课后习题既可用于自学,也可作为OpenMP和MPI小型实现项目的灵感来源,这类项目可以且应当配套任何严谨的并行计算课程。学生若能在并行计算系统上实际实现并仔细基准测试所建议的算法,将受益匪浅,这类并行计算系统可以且应当作为并行计算课程的配套资源。在课堂教学中,这些习题可作为作业和小型编程项目的基础,教师需为此提供足够的额外细节与精度要求。

英文摘要:

These lecture notes are designed to accompany an imaginary, virtual, undergraduate, one or two semester course on fundamentals of Parallel Computing as well as to serve as background and reference for graduate courses on High-Performance Computing, parallel algorithms and shared-memory multiprocessor programming. They introduce theoretical concepts and tools for expressing, analyzing and judging parallel algorithms and, in detail, cover the two most widely used concrete frameworks OpenMP and MPI as well as the threading interface pthreads for writing parallel programs for either shared or distributed memory parallel computers with emphasis on general concepts and principles. Code examples are given in a C-like style and many are actual, correct C code. The lecture notes deliberately do not cover GPU architectures and GPU programming, but the general concerns, guidelines and principles (time, work, cost, efficiency, scalability, memory structure and bandwidth) will be just as relevant for efficiently utilizing various GPU architectures. Likewise, the lecture notes focus on deterministic algorithms only and do not use randomization. The student of this material will find it instructive to take the time to understand concepts and algorithms visually. The exercises can be used for self-study and as inspiration for small implementation projects in OpenMP and MPI that can and should accompany any serious course on Parallel Computing. The student will benefit from actually implementing and carefully benchmarking the suggested algorithms on the parallel computing system that may or should be made available as part of such a Parallel Computing course. In class, the exercises can be used as basis for hand-ins and small programming projects for which sufficient, additional detail and precision should be provided by the instructor.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑