RecD：面向端到端深度学习推荐模型训练基础设施的去重方法
RecD: Deduplication for End-to-End Deep Learning Recommendation Model Training Infrastructure
AI总结：
针对工业级DLRM训练数据集特征重复带来的高开销问题，提出RecD端到端优化方案，通过优化数据生成流水线与新型IKJT张量格式实现特征去重，显著提升训练、预处理吞吐量与存储效率。
摘要：
我们提出RecD（Recommendation Deduplication，推荐去重），这是一套贯穿深度学习推荐模型（DLRM）训练流水线的端到端基础设施优化方案。工业级DLRM训练数据集中固有的特征重复会带来极高的存储、预处理和训练开销，RecD正是为解决这一问题而设计。特征重复的根源在于DLRM数据集由交互行为生成：每个用户会话可产生多条训练样本，但许多特征的取值在这些样本间保持不变。我们展示了RecD如何在已部署的训练流水线中，端到端地利用这一特性。RecD优化数据生成流水线，以降低数据集存储与预处理的资源需求，并最大化训练批次内的重复度。RecD提出了一种新的张量格式——InverseKeyedJaggedTensors（IKJTs，逆键控锯齿张量），用于对每个批次内的特征值进行去重。我们展示了DLRM模型架构如何利用IKJTs大幅提升训练吞吐量。在工业级DLRM训练系统中，RecD分别将训练吞吐量、预处理吞吐量和存储效率最高提升至2.48倍、1.79倍和3.71倍。
英文摘要：
We present RecD (Recommendation Deduplication), a suite of end-to-end infrastructure optimizations across the Deep Learning Recommendation Model (DLRM) training pipeline. RecD addresses immense storage, preprocessing, and training overheads caused by feature duplication inherent in industry-scale DLRM training datasets. Feature duplication arises because DLRM datasets are generated from interactions. While each user session can generate multiple training samples, many features' values do not change across these samples. We demonstrate how RecD exploits this property, end-to-end, across a deployed training pipeline. RecD optimizes data generation pipelines to decrease dataset storage and preprocessing resource demands and to maximize duplication within a training batch. RecD introduces a new tensor format, InverseKeyedJaggedTensors (IKJTs), to deduplicate feature values in each batch. We show how DLRM model architectures can leverage IKJTs to drastically increase training throughput. RecD improves the training and preprocessing throughput and storage efficiency by up to 2.48x, 1.79x, and 3.71x, respectively, in an industry-scale DLRM training system.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
