Bandana:使用非易失性存储器存储深度学习模型
Bandana: Using Non-volatile Memory for Storing Deep Learning Models
摘要:
典型的大规模推荐系统使用存储在大量DRAM上的深度学习模型。这些模型通常依赖于嵌入,这消耗了大部分所需的内存。我们提出了Bandana,一个存储系统,减少了嵌入式DRAM的足迹,通过使用非易失性存储器(NVM)作为主要存储介质,与少量的DRAM作为缓存。与DRAM相比,在NVM上存储嵌入的主要挑战是其有限的读取带宽。Bandana使用两种主要技术来解决这一限制:首先,它使用超图分区将可能一起读取的嵌入向量存储在同一物理位置,其次,它通过模拟数十个小型缓存来决定要在DRAM中缓存的嵌入向量的数量。这些技术使Bandana能够将NVM的有效读取带宽提高2- 3倍,从而显著降低总拥有成本。
英文摘要:
Typical large-scale recommender systems use deep learning models that are stored on a large amount of DRAM. These models often rely on embeddings, which consume most of the required memory. We present Bandana, a storage system that reduces the DRAM footprint of embeddings, by using Non-volatile Memory (NVM) as the primary storage medium, with a small amount of DRAM as cache. The main challenge in storing embeddings on NVM is its limited read bandwidth compared to DRAM. Bandana uses two primary techniques to address this limitation: first, it stores embedding vectors that are likely to be read together in the same physical location, using hypergraph partitioning, and second, it decides the number of embedding vectors to cache in DRAM by simulating dozens of small caches. These techniques allow Bandana to increase the effective read bandwidth of NVM by 2-3x and thereby significantly reduce the total cost of ownership.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。