arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2403.18682cs.DScs.DBcs.DC
收藏

JumpBackHash:告别取模运算,将键均匀分布到桶中

JumpBackHash: Say Goodbye to the Modulo Operation to Distribute Keys Uniformly to Buckets

Dynatrace Research(Dynatrace研究)

Otmar Ertl

AI总结:

本文提出一致性哈希算法 JumpBackHash,通过逆序生成活跃索引避免浮点运算并提升效率,实现键到桶的均匀稳定分布,可作为取模方法的替代方案。

摘要:

引言。分布式数据处理和存储系统需要高效的方法将键分布到各个桶中。传统的基于取模的映射虽然简单快速,但当桶的数量发生变化时不稳定,会导致系统资源利用率出现尖峰,例如网络或数据库请求。一致性哈希算法可以最小化重新映射,但要么显著更慢,要么需要浮点运算,要么基于标准库中很少提供的一族哈希函数。本文引入了 JumpBackHash,一种克服上述缺点的一致性哈希算法。方法。JumpBackHash 应用了借鉴自一致性加权采样的活跃索引概念,这天然带来一致性。它以逆序生成活跃索引,从而避免浮点运算,能够最小化所消耗的随机值数量并使用标准伪随机生成器,最终得到一个非常高效的算法。结果。理论分析表明,JumpBackHash 具有期望的常数运行时间。所消耗随机值数量的期望值和方差与实验完全一致。实证测试也证实了一致性。结论。JumpBackHash 为分布式系统中将键均匀分布到桶提供了一种快速高效的解决方案。其简单性、性能以及作为 Hash4j 开源库一部分的、可用于生产的 Java 实现,使其成为基于取模方法的可行替代方案,以改善分配和系统稳定性。

英文摘要:

Introduction. Distributed data processing and storage systems require efficient methods to distribute keys across buckets. While simple and fast, the traditional modulo-based mapping is unstable when the number of buckets changes, leading to spikes in system resource utilization, such as network or database requests. Consistent hash algorithms minimize remappings but are either significantly slower, require floating-point arithmetic, or are based on a family of hash functions rarely available in standard libraries. This work introduces JumpBackHash, a consistent hash algorithm that overcomes those shortcomings. Methodology. JumpBackHash applies the concept of active indices borrowed from consistent weighted sampling, which inherently leads to consistency. It generates the active indices in reverse order, which avoids floating-point operations, enables the minimization of consumed random values and the use of a standard pseudorandom generator, and finally leads to a very efficient algorithm. Results. Theoretical analysis shows that JumpBackHash has an expected constant runtime. The expected value and the variance of the number of consumed random values perfectly agree with the experiments. Empirical tests also confirm the consistency. Conclusion. JumpBackHash offers a fast and efficient solution for uniformly distributing keys across buckets in distributed systems. Its simplicity, performance, and the availability of a production-ready Java implementation as part of the Hash4j open source library make it a viable replacement for the modulo-based approach for improving assignment and system stability.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑