稀疏极端多标号排序树的快速推理
Enterprise-Scale Search: Accelerating Inference for Sparse Extreme Multi-Label Ranking Trees
摘要：
基于树的模型由于其次线性推理时间而支撑许多现代语义搜索引擎和推荐系统。在工业应用中，这些模型在极端规模下运行，其中每一点性能都至关重要。在极端尺度下的内存限制也要求模型是稀疏的，因此基于树的模型通常由稀疏矩阵代数例程后端处理。然而，目前还没有专门为稀疏结构设计的稀疏矩阵技术，在基于树的模型中遇到极端的多标签排名/分类（XMR/XMC）问题。为了解决这个问题，我们提出了屏蔽稀疏块乘法（MSCM）技术，一个稀疏矩阵技术专门为XMR树。MSCM易于实现，可并行化，并免费为任何现有的树推理流水线提供显着的性能提升。我们进行了一个全面的研究MSCM应用到几个不同的稀疏推理方案和基准我们的方法上的通用极端多标签排名框架。我们观察到MSCM在在线和批量推理设置、单线程和多线程设置以及许多不同的树模型和数据集上都提供了一致的显著加速。为了证明其在工业应用中的实用性，我们将MSCM应用于具有1亿种产品的企业级语义产品搜索问题，并在单个线程上实现了每次查询0.88 ms的亚毫秒延迟-延迟比香草推理技术减少了8倍。MSCM技术绝对不需要牺牲模型精度，因为它给出了与标准稀疏矩阵技术完全相同的结果。因此，我们相信MSCM将使XMR树的用户能够以非常小的成本在其推理管道中节省大量的计算资源。
英文摘要：
Tree-based models underpin many modern semantic search engines and recommender systems due to their sub-linear inference times. In industrial applications, these models operate at extreme scales, where every bit of performance is critical. Memory constraints at extreme scales also require that models be sparse, hence tree-based models are often back-ended by sparse matrix algebra routines. However, there are currently no sparse matrix techniques specifically designed for the sparsity structure one encounters in tree-based models for extreme multi-label ranking/classification (XMR/XMC) problems. To address this issue, we present the masked sparse chunk multiplication (MSCM) technique, a sparse matrix technique specifically tailored to XMR trees. MSCM is easy to implement, embarrassingly parallelizable, and offers a significant performance boost to any existing tree inference pipeline at no cost. We perform a comprehensive study of MSCM applied to several different sparse inference schemes and benchmark our methods on a general purpose extreme multi-label ranking framework. We observe that MSCM gives consistently dramatic speedups across both the online and batch inference settings, single- and multi-threaded settings, and on many different tree models and datasets. To demonstrate its utility in industrial applications, we apply MSCM to an enterprise-scale semantic product search problem with 100 million products and achieve sub-millisecond latency of 0.88 ms per query on a single thread -- an 8x reduction in latency over vanilla inference techniques. The MSCM technique requires absolutely no sacrifices to model accuracy as it gives exactly the same results as standard sparse matrix techniques. Therefore, we believe that MSCM will enable users of XMR trees to save a substantial amount of compute resources in their inference pipelines at very little cost.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
