CriteoTB的随机偏移块嵌入阵列（ROBE）基准MLPerf DLRM型号：1000 $\times$压缩和3.1 $\times$更快的推理
Random Offset Block Embedding Array (ROBE) for CriteoTB Benchmark MLPerf DLRM Model : 1000$\times$ Compression and 3.1$\times$ Faster Inference
摘要：
推荐数据的深度学习是近年来最普遍和最具挑战性的人工智能工作负载之一。最先进的推荐模型是与GPT-3和Switch Transformer等相匹配的最大模型之一。深度学习推荐模型（DLRM）的挑战来自于学习每个分类标记的密集嵌入。工业规模模型中的这些嵌入表可能高达数百TB。如此大的模型会带来过多的工程挑战，更不用说高昂的通信开销，以及较慢的训练和推理时间。其中，较慢的推理时间直接影响用户体验。DLRM的模型压缩正在获得越来越多的关注，社区最近展示了令人印象深刻的压缩结果。在本文中，我们提出了随机偏移块嵌入数组（ROBE）作为一个低内存替代嵌入表，提供了数量级的减少内存使用，同时保持准确性和提高执行速度。ROBE是一种简单的基本方法，可以提高缓存性能和随机哈希的方差，这本身就可以独立感兴趣。我们证明，我们可以成功地训练DLRM模型具有相同的精度，而使用$1000 \times$更少的内存。$1000\times$压缩模型直接导致更快的推理，而无需任何工程工作。特别是，我们表明，我们可以训练DLRM模型使用ROBE阵列的大小为100 MB的一个GPU上实现AUC为0.8025或更高的官方MLPerf CriteoTB基准DLRM模型的100 GB的要求，同时实现约$3.1\times$（209\%）的推理吞吐量的改善。
英文摘要：
Deep learning for recommendation data is one of the most pervasive and challenging AI workload in recent times. State-of-the-art recommendation models are one of the largest models matching the likes of GPT-3 and Switch Transformer. Challenges in deep learning recommendation models (DLRM) stem from learning dense embeddings for each of the categorical tokens. These embedding tables in industrial scale models can be as large as hundreds of terabytes. Such large models lead to a plethora of engineering challenges, not to mention prohibitive communication overheads, and slower training and inference times. Of these, slower inference time directly impacts user experience. Model compression for DLRM is gaining traction and the community has recently shown impressive compression results. In this paper, we present Random Offset Block Embedding Array (ROBE) as a low memory alternative to embedding tables which provide orders of magnitude reduction in memory usage while maintaining accuracy and boosting execution speed. ROBE is a simple fundamental approach in improving both cache performance and the variance of randomized hashing, which could be of independent interest in itself. We demonstrate that we can successfully train DLRM models with same accuracy while using $1000 \times$ less memory. A $1000\times$ compressed model directly results in faster inference without any engineering effort. In particular, we show that we can train DLRM model using ROBE array of size 100MB on a single GPU to achieve AUC of 0.8025 or higher as required by official MLPerf CriteoTB benchmark DLRM model of 100GB while achieving about $3.1\times$ (209\%) improvement in inference throughput.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
