寻找具有高性能和高可用性的关键价值存储
In Search of a Key Value Store with High Performance and High Availability
摘要:
近年来,写重的应用程序越来越普遍。如何有效地处理这种工作负载,是数据库系统领域深入研究的方向之一.写操作引起的开销主要由两个原因引起:1)硬件级,即,日志记录导致的IO开销。我们无法在短期内消除此成本2)双拷贝软件架构和串行重放。日志作为数据库体系结构的诞生就是为了克服软件的缺陷。但是,现有的日志数据库系统要么是建立在Infiniband或NVRam(Non-Volatile Random Access Memory)等特殊的基础设施之上,而这些基础设施还远没有得到广泛的应用,要么是借助Dynamo等其他系统来构建的,缺乏灵活性。为了支持我们的即时通讯业务,我们从零开始构建了一个只写一次的键值系统LogStore。LogStore的主要特性包括:1)每个分区一个线程的执行模式,消除了并发开销; 2)日志作为数据库,支持一次写入特性和备用数据库的刷新。通过嵌入复制协议而不依赖于其他基础设施来实现高可用性; 3)细粒度、低开销的数据缓冲池管理,有效地降低了IO开销。实验结果表明,LogStore在写操作、恢复和复制方面具有良好的性能
英文摘要:
In recent year, the write-heavy applications is more and more prevalent. How to efficiently handle this sort of workload is one of intensive research direction in the field of database system. The overhead caused by write operation is mainly issued by two reasons: 1) the hardware level, i.e., the IO cost caused by logging. We can't remove this cost in short period 2) the dual-copy software architecture and serial replay. The born of log as database architecture is originated to overcome the software defect. But existing systems treating log as database either are built on top of special infrastructure such as infiniband or NVRam (Non-Volatile Random access memory) which is far from widely available or are constructed with the help of other system such as Dynamo which is lack of flexibility. In this paper we build only write-once key-value system called LogStore from scratch to support our instant messenger business. The key features of LogStore include: 1) a single thread per partition executing mode, which eliminates the concurrency overhead; 2) log as database to enable write-once feature and freshness on the standby. We achieve high availability by embedding replication protocol other than dependent on other infrastructure; 3) fine-grained and low overhead data buffer pool management to effectively minimize IO cost. According to our empirical evaluations LogStore has good performance in write operation, recovery and replication
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。