DISTAL: The Distributed Tensor Algebra Compiler

I CAN HAS SUPERCOMPUTER? A Novel Approach to Teaching Parallel and Distributed Computing Concepts Using a Meme-Based Programming Language

A novel approach is presented to teach the parallel and distributed computing concepts of synchronization and remote memory access. The single program multiple data (SPMD) partitioned global address space (PGAS) model presented in this paper uses a procedural programming language appealing to undergraduate students. We propose that the amusing nature of the approach may engender creativity and interest using these concepts later in more sober environments. Specifically, we implement parallel extensions to LOLCODE within a source-to-source compiler sufficient for the development of parallel and…

From Clarity to Efficiency for Distributed Algorithms

This article describes a very high-level language for clear description of distributed algorithms and optimizations necessary for generating efficient implementations. The language supports high-level control flows where complex synchronization conditions can be expressed using high-level queries, especially logic quantifications, over message history sequences. Unfortunately, the programs would be extremely inefficient, including consuming unbounded memory, if executed straightforwardly. We present new optimizations that automatically transform complex synchronization conditions into incremen…

Implementing a Language for Distributed Systems: Choices and Experiences with Type Level and Macro Programming in Scala

Multitier programming languages reduce the complexity of developing distributed systems by developing the distributed system in a single coherent code base. The compiler or the runtime separate the code for the components of the distributed system, enabling abstraction over low level implementation details such as data representation, serialization and network protocols. Our ScalaLoci language allows developers to declare the different components and their architectural relation at the type level, allowing static reasoning about about distribution and remote communication and guaranteeing stat…

Cyclotron：将递推关系编译到分布式和脉动架构

Cyclotron: Compilation of Recurrences to Distributed and Systolic Architectures

Cyclotron是一个将递推方程表达的流数据流算法编译到分布式和脉动架构的框架与编译器，通过三层语言转换优化内存访问，支持用户定义调度实现流水线执行，在多硬件平台验证了可移植性，生成的分布式矩阵运算实现与ScaLAPACK竞争力相当。

Moderately Complex Paxos Made Simple: High-Level Executable Specification of Distributed Algorithms

This paper describes the application of a high-level language and method in developing simpler specifications of more complex variants of the Paxos algorithm for distributed consensus. The specifications are for Multi-Paxos with preemption, replicated state machine, and reconfiguration and optimized with state reduction and failure detection. The language is DistAlgo. The key is to express complex control flows and synchronization conditions precisely at a high level, using nondeterministic waits and message-history queries. We obtain complete executable specifications that are almost complete…

Clef编程语言中的共指数探索

Coexponential Exploration in the Clef Programming Language

该研究在并发编程语言Clef中开发有状态服务的共指数解释，证明相关转换的状态所有权与回复保留，实现分布式工作示例，将自动结构分析等关联，支持有状态服务的跨目标正确组合。

Applying Type Oriented Programming to the PGAS Memory Model

The Partitioned Global Address Space memory model has been popularised by a number of languages and applications. However this abstraction can often result in the programmer having to rely on some in built choices and with this implicit parallelism, with little assistance by the programmer, the scalability and performance of the code heavily depends on the compiler and choice of application. We propose an approach, type oriented programming, where all aspects of parallelism are encoded via types and the type system. The type information associated by the programmer will determine, for instance…

Breaking the Computation and Communication Abstraction Barrier in Distributed Machine Learning Workloads

Recent trend towards increasing large machine learning models require both training and inference tasks to be distributed. Considering the huge cost of training these models, it is imperative to unlock optimizations in computation and communication to obtain best performance. However, current logical separation between computation and communication kernels in deep learning frameworks misses the optimization opportunities across such barrier. Breaking this abstraction with a holistic consideration can provide many optimizations to provide performance improvements in distributed workloads. Manua…

A Type System for the Automatic Distribution of Higher-order Synchronous Dataflow Programs

We address the design of distributed systems with synchronous dataflow programming languages. As modular design entails handling both architectural and functional modularity, our first contribution is to extend an existing synchronous dataflow programming language with primitives allowing the description of a distributed architecture and the localization of some expressions onto some processors. We also present a distributed semantics to formalize the distributed execution of synchronous programs. Our second contribution is to provide a type system, in order to infer the localization of non-an…

MCBeth: A Measurement Based Quantum Programming Language

Gate-based quantum programming languages are ubiquitous but measurement-based languages currently exist only on paper. This work introduces MCBeth, a quantum programming language which allows programmers to directly represent, program, and simulate measurement-based and cluster state computation by building upon the measurement calculus. While MCBeth programs are meant to be executed directly on hardware, to take advantage of current machines we also provide a compiler to gate-based instructions. We argue that there are clear advantages to measurement-based quantum computation compared to gate…

在云上扩展过程间数据流分析

Scaling Inter-procedural Dataflow Analysis on the Cloud

本文提出分布式分析框架BigDataflow，基于大规模图处理思想设计分布式工作列表算法，支持过程间数据流分析，能在几分钟内分析数百万行代码，显著提升分析效率。

用状态变换子实现编排程序与霍尔逻辑的机械化

Mechanizing Choreographic Programs and Hoare Logic with State Transformers

该研究将状态变换子模型引入编排编程，在Lean中机械化实现支持多种通信与方法的编排语言，证明了端点投影等性质及霍尔逻辑，规避了绑定替换问题以简化机械化。

SparseLNR: Accelerating Sparse Tensor Computations Using Loop Nest Restructuring

Sparse tensor algebra computations have become important in many real-world applications like machine learning, scientific simulations, and data mining. Hence, automated code generation and performance optimizations for tensor algebra kernels are paramount. Recent advancements such as the Tensor Algebra Compiler (TACO) greatly generalize and automate the code generation for tensor algebra expressions. However, the code generated by TACO for many important tensor computations remains suboptimal due to the absence of a scheduling directive to support transformations such as distribution/fusion. …