基于依赖日志的分布式事务处理和恢复的扩展
Scaling Distributed Transaction Processing and Recovery based on Dependency Logging
摘要:
DGCC协议已被证明可以在多核内存系统上实现良好的性能。然而,分布式事务复杂的依赖性解决,因此,一个有效的事务划分策略是必不可少的,以减少昂贵的多节点分布式事务。在故障恢复期间,必须从最后一个检查点开始检查日志,并根据受影响的事务的分区和执行方式重新执行这些事务。现有的方法将事务管理和恢复视为两个独立的问题,即使恢复取决于事务执行的顺序。在本文中,我们建议将事务管理和恢复问题视为一个问题。我们首先提出了一种高效的基于分布式依赖关系图的并发控制(DistDGCC)协议,用于处理跨多个节点的事务,并提出了一种新型高效的日志记录协议,称为依赖关系日志记录,该协议也利用依赖关系图进行高效的日志记录和恢复。DistDGCC通过批量处理事务来优化每个分布式事务的平均成本。此外,它还减少了由分布式事务引起的线程阻塞的影响,从而提高了运行时性能。此外,依赖性日志记录利用与DistDGCC所使用的相同的数据结构来减少日志记录开销,以及利用逻辑依赖性信息来提高恢复并行性。进行了大量的实验,以评估我们所提出的技术对国家的最先进的技术的性能。实验结果表明,DistDGCC是有效的和可扩展的,依赖日志支持快速恢复与边际运行时开销。因此,整体系统性能作为结果被显著地改进。
英文摘要:
DGCC protocol has been shown to achieve good performance on multi-core in-memory system. However, distributed transactions complicate the dependency resolution, and therefore, an effective transaction partitioning strategy is essential to reduce expensive multi-node distributed transactions. During failure recovery, log must be examined from the last checkpoint onwards and the affected transactions are re-executed based on the way they are partitioned and executed. Existing approaches treat both transaction management and recovery as two separate problems, even though recovery is dependent on the sequence in which transactions are executed. In this paper, we propose to treat the transaction management and recovery problems as one. We first propose an efficient Distributed Dependency Graph based Concurrency Control (DistDGCC) protocol for handling transactions spanning multiple nodes, and propose a new novel and efficient logging protocol called Dependency Logging that also makes use of dependency graphs for efficient logging and recovery. DistDGCC optimizes the average cost for each distributed transaction by processing transactions in batch. Moreover, it also reduces the effects of thread blocking caused by distributed transactions and consequently improves the runtime performance. Further, dependency logging exploits the same data structure that is used by DistDGCC to reduce the logging overhead, as well as the logical dependency information to improve the recovery parallelism. Extensive experiments are conducted to evaluate the performance of our proposed technique against state-of-the-art techniques. Experimental results show that DistDGCC is efficient and scalable, and dependency logging supports fast recovery with marginal runtime overhead. Hence, the overall system performance is significantly improved as a result.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。