首页/论文/2002.08316

arXiv 2002.083162020-02cs.PF

Re-evaluating scaling methods for distributed parallel systems

János Végh

备注：: 12 pages, 8 figures. arXiv admin note: text overlap with arXiv:2001.01266

英文摘要：

The paper explains why Amdahl's Law shall be interpreted specifically for distributed parallel systems and why it generated so many debates, discussions, and abuses. We set up a general model and list many of the terms affecting parallel processing. We scrutinize the validity of neglecting certain terms in different approximations, with special emphasis on the famous scaling laws of parallel processing. We clarify that when using the right interpretation of terms, Amdahl's Law is the governing law of all kinds of parallel processing. Amdahl's Law describes among others the history of supercomputing, the inherent performance limitation of the different kinds of parallel processing and it is the basic Law of the 'modern computing' paradigm, that the computing systems working under extreme computing conditions are desperately needed.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

DuctTeip: An efficient programming model for distributed task based parallel computing

Current high-performance computer systems used for scientific computing typically combine shared memory computational nodes in a distributed memory environment. Extracting high performance from these complex systems requires tailored approaches. Task based parallel programming has been successful both in simplifying the programming and in exploiting the available hardware parallelism for shared memory systems. In this paper we focus on how to extend task parallel programming to distributed memory systems. We use a hierarchical decomposition of tasks and data in order to accommodate the differe…

2018-01-10cs.CEcs.DCcs.PF

应用于分布式量子计算的并行量子纠错码解码

Parallel QEC Decoding Applied to Distributed Quantum Computing

University of Parma（帕尔马大学）

研究针对分布式量子计算提出并行QEC解码新方法，核心是用奇异值分解预处理信念传播所得错误向量，经复杂度、准确性和可扩展性评估，为分布式量子计算的QEC解码提供了新途径。

2026-07-10cs.DCcs.PFquant-ph

Multicore-aware parallel temporal blocking of stencil codes for shared and distributed memory

New algorithms and optimization techniques are needed to balance the accelerating trend towards bandwidth-starved multicore chips. It is well known that the performance of stencil codes can be improved by temporal blocking, lessening the pressure on the memory interface. We introduce a new pipelined approach that makes explicit use of shared caches in multicore environments and minimizes synchronization and boundary overhead. For clusters of shared-memory nodes we demonstrate how temporal blocking can be employed successfully in a hybrid shared/distributed-memory environment.

2009-12-22cs.DCcs.PF

Distributed dynamic load balancing for task parallel programming

In this paper, we derive and investigate approaches to dynamically load balance a distributed task parallel application software. The load balancing strategy is based on task migration. Busy processes export parts of their ready task queue to idle processes. Idle--busy pairs of processes find each other through a random search process that succeeds within a few steps with high probability. We evaluate the load balancing approach for a block Cholesky factorization implementation and observe a reduction in execution time on the order of 5\% in the selected test cases.

2018-01-14cs.DCcs.MScs.PF

FaaSter Troubleshooting——评估无服务器应用的分布式追踪方法

FaaSter Troubleshooting -- Evaluating Distributed Tracing Approaches for Serverless Applications

针对无服务器应用故障排查难、平台观测支持不足的问题，研究者提出故障可观测性表征模型，对比开发者驱动与平台支持两种分布式追踪方案的性能权衡，为无服务器架构故障排查提供新参考。

2021-10-07cs.PFcs.SE

评估分布式文件系统中的容错性和可扩展性：GFS、HDFS 和 MinIO 的案例研究

Evaluating Fault Tolerance and Scalability in Distributed File Systems: A Case Study of GFS, HDFS, and MinIO

本文评估GFS、HDFS和MinIO三种分布式文件系统的容错性与可扩展性，通过分析数据冗余、故障处理和访问协议，为企业选择合适系统提供实用见解。

2025-02-28cs.DCcs.ETcs.PF

更多相关论文收起更多论文

Parallel Breadth-First Search on Distributed Memory Systems

Data-intensive, graph-based computations are pervasive in several scientific applications, and are known to to be quite challenging to implement on distributed memory systems. In this work, we explore the design space of parallel algorithms for Breadth-First Search (BFS), a key subroutine in several graph algorithms. We present two highly-tuned parallel approaches for BFS on large parallel systems: a level-synchronous strategy that relies on a simple vertex-based partitioning of the graph, and a two-dimensional sparse matrix-partitioning-based approach that mitigates parallel communication ove…

2011-10-14cs.DCcs.MScs.PF

Crossing the Architectural Barrier: Evaluating Representative Regions of Parallel HPC Applications

Exascale computing will get mankind closer to solving important social, scientific and engineering problems. Due to high prototyping costs, High Performance Computing (HPC) system architects make use of simulation models for design space exploration and hardware-software co-design. However, as HPC systems reach exascale proportions, the cost of simulation increases, since simulators themselves are largely single-threaded. Tools for selecting representative parts of parallel applications to reduce running costs are widespread, e.g., BarrierPoint achieves this by analysing, in simulation, abstra…

2018-03-20cs.ARcs.DCcs.PF

Evaluating Load Balancing Performance in Distributed Storage with Redundancy

To facilitate load balancing, distributed systems store data redundantly. We evaluate the load balancing performance of storage schemes in which each object is stored at $d$ different nodes, and each node stores the same number of objects. In our model, the load offered for the objects is sampled uniformly at random from all the load vectors with a fixed cumulative value. We find that the load balance in a system of $n$ nodes improves multiplicatively with $d$ as long as $d = o\left(\log(n)\right)$, and improves exponentially once $d = Θ\left(\log(n)\right)$. We show that the load balance impr…

2021-01-23cs.ITcs.PFmath.IT

Diversity/Parallelism Trade-off in Distributed Systems with Redundancy

As numerous machine learning and other algorithms increase in complexity and data requirements, distributed computing becomes necessary to satisfy the growing computational and storage demands, because it enables parallel execution of smaller tasks that make up a large computing job. However, random fluctuations in task service times lead to straggling tasks with long execution times. Redundancy, in the form of task replication and erasure coding, provides diversity that allows a job to be completed when only a subset of redundant tasks is executed, thus removing the dependency on the straggli…

2021-12-18cs.DCcs.ITcs.PF