MCPrioQ:一种面向在线稀疏马尔可夫链的无锁算法
MCPrioQ: A lock-free algorithm for online sparse markov-chains
AI总结:
针对高性能系统中超大规模图内存与计算效率难兼顾的问题,提出无锁稀疏马尔可夫链数据结构MCPrioQ,通过哈希表、原子指令及改进的读-复制-更新机制实现低复杂度的在线学习与近似正确的并发查找,适配推荐系统场景。
摘要:
在高性能系统中,有时难以构建在内存和计算上都高效的超大规模图。本文提出了一种名为Markov-chain-priority-queue(马尔可夫链优先队列,简称MCPrioQ)的数据结构,它是一种无锁稀疏马尔可夫链,支持在线持续学习,更新的时间复杂度为$O(1)$,推理的时间复杂度为$O(CDF^{-1}(t))$。MCPrioQ特别适用于推荐系统中按概率降序查找$n$个物品的场景。它通过哈希表和原子指令实现并发更新,通过一种新型优先队列实现查找,即使在并发更新期间也能得到近似正确的结果。其近似正确性和无锁特性由读-复制-更新机制维持,但该机制的语义略有调整,允许元素交换而非传统的弹出-插入方案。
英文摘要:
In high performance systems it is sometimes hard to build very large graphs that are efficient both with respect to memory and compute. This paper proposes a data structure called Markov-chain-priority-queue (MCPrioQ), which is a lock-free sparse markov-chain that enables online and continuous learning with time-complexity of $O(1)$ for updates and $O(CDF^{-1}(t))$ inference. MCPrioQ is especially suitable for recommender-systems for lookups of $n$-items in descending probability order. The concurrent updates are achieved using hash-tables and atomic instructions and the lookups are achieved through a novel priority-queue which allows for approximately correct results even during concurrent updates. The approximatly correct and lock-free property is maintained by a read-copy-update scheme, but where the semantics have been slightly updated to allow for swap of elements rather than the traditional pop-insert scheme.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。