Consensus protocols are currently the bottlenecks that prevent blockchain systems from scaling. However, we argue that transaction execution is also important to the performance and security of blockchains. In other words, there are ample opportunities to speed up and further secure blockchains by reducing the cost of transaction execution.
Our goal is to understand how much we can speed up blockchains by exploiting transaction concurrency available in blockchain workloads. To this end, we first analyze historical data of seven major public blockchains, namely Bitcoin, Bitcoin Cash, Litecoin, Dogecoin, Ethereum, Ethereum Classic, and Zilliqa. We consider two metrics for concurrency, namely the single-transaction conflict rate per block, and the group conflict rate per block. We find that there is more concurrency in UTXO-based blockchains than in account-based ones, although the amount of concurrency in the former is lower than expected. Another interesting finding is that some blockchains with larger blocks have more concurrency than blockchains with smaller blocks. Next, we propose an analytical model for estimating the transaction execution speed-up given an amount of concurrency. Using results from our empirical analysis, the model estimates that 6x speed-ups in Ethereum can be achieved if all available concurrency is exploited.
The crux of software transactional memory (STM) is to combine an easy-to-use programming interface with an efficient utilization of the concurrent-computing abilities provided by modern machines. But does this combination come with an inherent cost? We evaluate the cost of concurrency by measuring the amount of expensive synchronization that must be employed in an STM implementation that ensures positive concurrency, i.e., allows for concurrent transaction processing in some executions. We focus on two popular progress conditions that provide positive concurrency: progressiveness and permissiv…
To utilize the multi-core processors properly concurrent programming is needed. Concurrency control is the main challenge while designing a correct and efficient concurrent program. Software Transactional Memory Systems (STMs) provides ease of multithreading to the programmer without worrying about concurrency issues such as deadlock, livelock, priority inversion, etc. Most of the STMs works on read-write operations known as RWSTMs. Some STMs work at high-level operations and ensure greater concurrency than RWSTMs. Such STMs are known as Object-Based STMs (OSTMs). The transactions of OSTMs can…
State-of-the-art \emph{software transactional memory (STM)} implementations achieve good performance by carefully avoiding the overhead of \emph{incremental validation} (i.e., re-reading previously read data items to avoid inconsistency) while still providing \emph{progressiveness} (allowing transactional aborts only due to \emph{data conflicts}). Hardware transactional memory (HTM) implementations promise even better performance, but offer no progress guarantees. Thus, they must be combined with STMs, leading to \emph{hybrid} TMs (HyTMs) in which hardware transactions must be \emph{instrument…
StableShard: Stable and Scalable Blockchain Sharding with High Concurrency via Collaborative Committees
Great Bay University（大湾区大学）；Institute of High Performance Computing, A*STAR（A*STAR高性能计算研究所）；Southern University of Science and Technology（南方科技大学）；Peng Cheng Laboratory（鹏城实验室）
Improving transaction throughput is one of the main challenges in decentralized payment systems. Attempts to improve transaction throughput in cryptocurrencies are usually a trade-off between throughput and security or introduce a central component. We propose Hydra, a decentralized protocol that improves transaction throughput without the security trade-off and has no central component. Our novel approach distributes blocks over multiple blockchains. Hydra makes a trade-off between transaction throughput and finality, the time it takes to stabilize the record of a transaction in the blockchai…
The concurrency control algorithms in transactional systems limits concurrency to provide strong semantics, which leads to poor performance under high contention. As a consequence, many transactional systems eschew strong semantics to achieve acceptable performance. We show that by leveraging semantic information associated with the transactional programs to increase concurrency, it is possible to significantly improve performance while maintaining linearizability. To this end, we introduce the lazy state determination API to easily expose the semantics of application transactions to the datab…
Public blockchains like Ethereum use Merkle trees to verify transactions received from untrusted servers before applying them to the blockchain. We empirically show that the low throughput of such blockchains is due to the I/O bottleneck associated with using Merkle trees for processing transactions. We present RAINBLOCK, a new architecture for public blockchains that increases throughput without affecting security. RAINBLOCK achieves this by tackling the I/O bottleneck on two fronts: first, decoupling transaction processing from I/O, and removing I/O from the critical path; second, reducing I…
Performance and scalability are major concerns for blockchains: permissionless systems are typically limited by slow proof of X consensus algorithms and sequential post-order transaction execution on every node of the network. By introducing a small amount of trust in their participants, permissioned blockchain systems such as Hyperledger Fabric can benefit from more efficient consensus algorithms and make use of parallel pre-order execution on a subset of network nodes. Fabric, in particular, has been shown to handle tens of thousands of transactions per second. However, this performance is o…
While many researchers adopt a sharding approach to design scaling blockchains, few works have studied the transaction placement problem incurred by sharding protocols. The widely-used hashing placement algorithm renders an overwhelming portion of transactions as cross-shard. In this paper, we analyze the high cost of cross-shard transactions and reveal that most Bitcoin transactions have simple dependencies and can become single-shard under a placement algorithm taking transaction dependencies into account. In addition, we perform a case study of OptChain, which is the state-of-the-art transa…
Hyperledger Fabric is a popular permissioned blockchain system that features a highly modular and extensible system for deploying permissioned blockchains which are expected to have a major effect on a wide range of sectors. Unlike traditional blockchain systems such as Bitcoin and Ethereum, Hyperledger Fabric uses the EOV model for transaction processing: the submitted transactions are executed by the endorsing peer, ordered and batched by the ordering services, and validated by the validating peers. Due to this EOV workflow, a well-documented issue that arises is the multi-version concurrenc…
We use historical data to estimate the potential benefit of speculative techniques for executing Ethereum smart contracts in parallel. We replay transaction traces of sampled blocks from the Ethereum blockchain over time, using a simple speculative execution engine. In this engine, miners attempt to execute all transactions in a block in parallel, rolling back those that cause data conflicts. Aborted transactions are then executed sequentially. Validators execute the same schedule as miners. We find that our speculative technique yields estimated speed-ups starting at about 8-fold in 2016, dec…
Sharding is used to address the performance and scalability issues of the blockchain protocols, which divides the overall transaction processing costs among multiple clusters of nodes. Shards require less storage capacity and communication and computation cost per node than the existing whole blockchain networks, and they operate in parallel to maximize performance. However, existing sharding solutions use locks for transaction isolation which lowers the system throughput and may introduce deadlocks. In this paper, we propose a lockless transaction method for ensuring transaction isolation wit…
Despite the success in various scenarios, blockchain systems, especially EVM-compatible ones that serially execute transactions, still face the significant challenge of limited throughput. Concurrent transaction execution is a promising technique to accelerate transaction processing and increase the overall throughput. Existing concurrency control algorithms, however, fail to obtain enough speedups in real-world blockchains due to the high-contention workloads. In this paper, we propose a novel operation-level concurrency control algorithm designed for blockchains. The core idea behind our alg…
Transactional Memory (TM) is an approach aiming to simplify concurrent programming by automating synchronization while maintaining efficiency. TM usually employs the optimistic concurrency control approach, which relies on transactions aborting and restarting if conflicts occur. However, an aborted transaction can still leave some effects in the system that cannot be cleaned up, if irrevocable operations are present within its code. The pessimistic approach eliminates that problem, since it relies on deferring operations in case of conflict rather than aborting, but hitherto pessimistic TMs su…
Blockchains have a two-sided reputation: they are praised for disrupting some of our institutions through innovative technology for good, yet notorious for being slow and expensive to use. In this work, we tackle this issue with concurrency, yet we aim to take a radically different approach by valuing simplicity. We embrace the simplicity through two steps: first, we formulate a simple runtime mechanism to deal with conflicts called concurrency delegation. This method is much simpler and has less overhead, particularly in scenarios where conflicting transactions are relatively rare. Moreover, …
Contemporary blockchain such as Bitcoin and Ethereum execute transactions serially by miners and validators and determine the Proof-of-Work (PoW). Such serial execution is unable to exploit modern multi-core resources efficiently, hence limiting the system throughput and increasing the transaction acceptance latency. The objective of this work is to increase the transaction throughput by introducing parallel transaction execution using a static analysis technique. We propose a framework DiPETrans for the distributed execution of the transactions in a block. Here, peers in the blockchain networ…
Since the creation of Bitcoin in 2009 we have seen a great push towards public and private blockchains. In order to avoid fragmentation, a global network connecting all these blockchains is envisioned. Just like the Internet facilitates communication and the transfer of information, we propose a system, similar in size and reach for payments and transactions: A cryptographically-secure off-chain multi-asset instant transaction network (COMIT) can connect and exchange any asset on any blockchain to any other blockchain using a cross-chain routing protocol (CRP). COMIT is a super blockchain netw…
Massively scalable web applications encounter a fundamental tension in computing between "performance" and "correctness": performance is often addressed by using a large and therefore distributed machine where programs are multi-threaded and interruptible, whereas correctness requires data invariants to be maintained with certainty. A solution to this problem is "transactions" [Gray-Reuter]. Some distributed systems such as Google App Engine [http://code.google.com/appengine/docs/] provide transaction semantics but only for functions that access one of a set of predefined local regions of the …
This paper proposes an efficient framework to execute Smart Contract Transactions (SCTs) concurrently based on object semantics, using optimistic Single-Version Object-based Software Transactional Memory Systems (SVOSTMs) and Multi-Version OSTMs (MVOSTMs). In our framework, a multi-threaded miner constructs a Block Graph (BG), capturing the object-conflicts relations between SCTs, and stores it in the block. Later, validators re-execute the same SCTs concurrently and deterministically relying on this BG. A malicious miner can modify the BG to harm the blockchain, e.g., to cause double-spending…
The cross-blockchain transaction remains one of the most challenging problems in blockchains. The root cause of the challenge lies in the nondeterministic nature of blockchains: A $n$-party transaction across multiple blockchains might be partially rolled back due to the potential forks in any of the participating blockchains---eventually, only one fork will survive in the competition among miners. While some effort has recently been made to developing hierarchically distributed commit protocols to make multi-party transactions progress, there is no systematic method to reason about the transa…
We present SSS, a scalable transactional key-value store deploying a novel distributed concurrency control that provides external consistency for all transactions, never aborts read-only transactions due to concurrency, all without specialized hardware. SSS ensures the above properties without any centralized source of synchronization. SSS's concurrency control uses a combination of vector clocks and a new technique, called snapshot-queuing, to establish a single transaction serialization order that matches the order of transaction completion observed by clients. We compare SSS against high pe…
The interoperability across multiple blockchains would play a critical role in future blockchain-based data management paradigm. Existing techniques either work only for two blockchains or requires a centralized component to govern the cross-blockchain transaction execution, neither of which would meet the scalability requirement. This paper proposes a new distributed commit protocol, namely \textit{cross-blockchain transaction} (CBT), for conducting transactions across an arbitrary number of blockchains without any centralized component. The key idea of CBT is to extend the two-phase commit p…
Transactive microgrids are emerging as a transformative solution for the problems faced by distribution system operators due to an increase in the use of distributed energy resources and a rapid acceleration in renewable energy generation, such as wind and solar power. Distributed ledgers have recently found widespread interest in this domain due to their ability to provide transactional integrity across decentralized computing nodes. However, the existing state of the art has not focused on the privacy preservation requirement of these energy systems -- the transaction level data can provide …
Within the last few years, a countless number of blockchain systems have emerged on the market, each one claiming to revolutionize the way of distributed transaction processing in one way or the other. Many blockchain features, such as byzantine fault tolerance (BFT), are indeed valuable additions in modern environments. However, despite all the hype around the technology, many of the challenges that blockchain systems have to face are fundamental transaction management problems. These are largely shared with traditional database systems, which have been around for decades already. These simil…
Synchronization of transaction pools (mempools) has shown potential for improving the performance and block propagation delay of state-of-the-art blockchains. Indeed, various heuristics have been proposed in the literature to this end, all of which incorporate exchanges of unconfirmed transactions into their block propagation protocol. In this work, we take a different approach, maintaining transaction synchronization outside (and independently) of the block propagation channel. In the process, we formalize the synchronization problem within a graph theoretic framework and introduce a novel al…
Public blockchains are decentralized networks where each participating node executes the same decision-making process. This form of decentralization does not scale well because the same data are stored on each network node, and because all nodes must validate each transaction prior to their confirmation. One solution approach decomposes the nodes of a blockchain network into subsets called "shards", each shard processing and storing disjoint sets of transactions in parallel. To fully benefit from the parallelism of sharded blockchains, the processing load of shards must be evenly distributed. …
In Bitcoin and Ethereum, nodes require large storage capacity to maintain all the blockchain data, such as transactions, UTXOs, and account states. As of May 2020, the storage size of the Bitcoin blockchain has expanded to 270 GB, and it will continue to increase. This storage requirement is a major hurdle to becoming a block proposer or validator. Although many studies have attempted to reduce the storage size, in the proposed methods, a node cannot keep all blocks or cannot generate a block. We propose an architecture called Trail that allows nodes to hold all blocks in a small storage and t…
Decentralized control, low-complexity, flexible and efficient communications are the requirements of an architecture that aims to scale blockchains beyond the current state. Such properties are attainable by reducing ledger size and providing parallel operations in the blockchain. Sharding is one of the approaches that lower the burden of the nodes and enhance performance. However, the current solutions lack the features for resolving concurrency during cross-shard communications. With multiple participants belonging to different shards, handling concurrent operations is essential for optimal …
Traditional public blockchain systems typically had very limited transaction throughput because of the bottleneck of the consensus protocol itself. With recent advances in consensus technology, the performance limit has been greatly lifted, typically to thousands of transactions per second. With this, transaction execution has become a new performance bottleneck. Exploiting parallelism in transaction execution is a clear and direct way to address this and to further increase transaction throughput. Although some recent literature introduced concurrency control mechanisms to execute smart contr…
In this paper, we design and implement the first-ever decentralized replicated relational database with blockchain properties that we term blockchain relational database. We highlight several similarities between features provided by blockchain platforms and a replicated relational database, although they are conceptually different, primarily in their trust model. Motivated by this, we leverage the rich features, decades of research and optimization, and available tooling in relational databases to build a blockchain relational database. We consider a permissioned blockchain model of known, bu…
The interoperability across multiple or many blockchains would play a critical role in the forthcoming blockchain-based data management paradigm. In particular, how to ensure the ACID properties of those transactions across an arbitrary number of blockchains remains an open problem in both academic and industry: Existing solutions either work for only two blockchains or requires a centralized component, neither of which would meet the scalability requirement in practice. This short paper shares our vision and some early results toward scalable cross-blockchain transactions. Specifically, we de…
Software Transactional Memory systems (STMs) have garnered significant interest as an elegant alternative for addressing synchronization and concurrency issues with multi-threaded programming in multi-core systems. Client programs use STMs by issuing transactions. STM ensures that transaction either commits or aborts. A transaction aborted due to conflicts is typically re-issued with the expectation that it will complete successfully in a subsequent incarnation. However, many existing STMs fail to provide starvation freedom, i.e., in these systems, it is possible that concurrency conflicts may…
The state-of-the-art techniques for processing cross-blockchain transactions take a simple centralized approach: when the assets on blockchain $X$, say $X$-coins, are exchanged with the assets on blockchain $Y$---the $Y$-coins, those $X$-coins need to be exchanged to a "middle" medium (such as Bitcoin) that is then exchanged to $Y$-coins. If there are more than two parties involved in a single global transaction, the global transaction is split into multiple local two-party transactions, each of which follows the above central-exchange protocol. Unfortunately, the atomicity of the global trans…
The cross-blockchain transaction (CBT) serves as a cornerstone for the next-generation, blockchain-based data management systems. However, state-of-the-art CBT models do not address the effect of the possible local fork suspension that might invalidate the entire CBT. This paper takes an algebraic-topological approach to abstract the blockchains and their transactions into simplicial complexes and shows that CBTs cannot complete in either a \textit{committed} or an \textit{aborted} status by a $t$-resilient message-passing protocol. This result implies that a more sophisticated model is in nee…
This paper presents TXSC, a framework that provides smart contract developers with transaction primitives. These primitives allow developers to write smart contracts without the need to reason about the anomalies that can arise due to concurrent smart contract function executions.
Traditionally, blockchain systems involve sharing transaction information across all blockchain network participants. Clearly, this introduces barriers to the adoption of the technology by the enterprise world, where preserving the privacy of the business data is a necessity. Previous efforts to bring privacy and blockchains together either still leak partial information, are restricted in their functionality or use costly mechanisms like zk-SNARKs. In this paper, we propose the Multi-Shard Private Transaction (MSPT) protocol, a novel privacy-preserving protocol for permissioned blockchains, w…
A blockchain is an append-only linked-list of blocks, which is maintained at each participating node. Each block records a set of transactions and their associated metadata. Blockchain transactions act on the identical ledger data stored at each node. Blockchain was first perceived by Satoshi Nakamoto as a peer-to-peer digital-commodity (also known as crypto-currency) exchange system. Blockchains received traction due to their inherent property of immutability-once a block is accepted, it cannot be reverted.
In this paper we are interested in bounding the number of instructions taken to process transactions. The main result is a multiversion transactional system that supports constant delay (extra instructions beyond running in isolation) for all read-only transactions, delay equal to the number of processes for writing transactions that are not concurrent with other writers, and lock-freedom for concurrent writers. The system supports precise garbage collection in that versions are identified for collection as soon as the last transaction releases them. As far as we know these are first results t…
Many existing blockchains do not adequately address all the characteristics of distributed system applications and suffer from serious architectural limitations resulting in performance and confidentiality issues. While recent permissioned blockchain systems, have tried to overcome these limitations, their focus has mainly been on workloads with no-contention, i.e., no conflicting transactions. In this paper, we introduce OXII, a new paradigm for permissioned blockchains to support distributed applications that execute concurrently. OXII is designed for workloads with (different degrees of) co…
Attacks targeting several millions of non-internet based application users are on the rise. These applications such as SMS and USSD typically do not benefit from existing multi-factor authentication methods due to the nature of their interaction interfaces and mode of operations. To address this problem, we propose an approach that augments blockchain with multi-factor authentication based on evidence from blockchain transactions combined with risk analysis. A profile of how a user performs transactions is built overtime and is used to analyse the risk level of each new transaction. If a trans…
Many classical blockchains are known to have an embarrassingly low transaction throughput, down to Bitcoin's notorious seven transactions per second limit.Various proposals and implementations for increasing throughput emerged in the first decade of blockchain research. But how much concurrency is possible? In their early days, blockchains were mostly used for simple transfers from user to user. More recently, however, decentralized finance (DeFi) and NFT marketplaces have completely changed what is happening on blockchains. Both are built using smart contracts and have gained significant popu…
Blockchain has been considered as an important technique to enable secure management of virtual network functions and network slices. To understand such capabilities of a blockchain, e.g. transaction confirmation time, demands a thorough study on the transaction characteristics of the blockchain. This paper presents a comprehensive study on the transaction characteristics of Bitcoin -- the first blockchain application, focusing on the underlying fundamental processes. A set of results and findings are obtained, which provide new insight into understanding the transaction and traffic characteri…
With the rise of digital currency systems that rely on blockchain to ensure ledger security, the ability to perform cross-chain transactions is becoming a crucial interoperability requirement. Such transactions allow not only funds to be transferred from one blockchain to another (as done in atomic swaps), but also a blockchain to verify the inclusion of any event on another blockchain. Cross-chain bridges are protocols that allow on-chain exchange of cryptocurrencies, on-chain transfer of assets to sidechains, and cross-shard verification of events in sharded blockchains, many of which rely o…
Scalability is one of the main roadblocks to business adoption of blockchain systems. Despite recent intensive research on using sharding techniques to enhance the scalability of blockchain systems, existing solutions do not efficiently address cross-shard transactions. In this paper, we introduce SharPer, a permissioned blockchain system that improves scalability by clustering (partitioning) the nodes and assigning different data shards to different clusters where each data shard is replicated on the nodes of a cluster. SharPer supports both intra-shard and cross-shard transactions and proces…
A major challenge in blockchain sharding protocols is that more than 95% transactions are cross-shard. Not only those cross-shard transactions degrade the system throughput but also double the confirmation time, and exhaust an already scarce network bandwidth. Are cross-shard transactions imminent for sharding schemes? In this paper, we propose a new sharding paradigm, called OptChain, in which cross-shard transactions are minimized, resulting in almost twice faster confirmation time and throughput. By treating transactions as a stream of nodes in an online graph, OptChain utilizes a lightweig…
