arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1909.05182cs.PF
收藏

Sentinel:面向深度学习的异构主MemorySystems数据管理

Sentinel: Runtime Data Management on Heterogeneous Main MemorySystems for Deep Learning

Jie Ren, Jiaolin Luo, Kai Wu, Minjia Zhang, Dong Li

摘要:

软件管理的异构存储器(HM)提供了一个很有前途的解决方案,以增加存储容量和成本效率。然而,要释放HM的性能潜力,我们面临着数据管理的问题。给定一个具有不同执行阶段的应用程序,并且每个阶段都可能具有不同的工作集,我们必须在HM的内存组件之间移动数据以优化性能。深度神经网络(DNN)作为数据中心的常见工作负载,给HM的数据管理带来了巨大的挑战。这种工作负载通常采用任务并行执行模型,具有大量小数据对象和细粒度操作(任务)的特点。这种执行模型对内存分析和有效的数据迁移提出了挑战。我们介绍了Sentinel,一个自动优化数据迁移的运行时系统(即,数据管理)以实现与具有小得多的快速存储器容量的仅快速存储器系统上的性能类似的性能。为了实现这一目标,Sentinel利用深度学习的领域知识,采用自定义方法进行数据管理。Sentinel利用工作负载可重复性来打破性能分析准确性和开销之间的困境;它通过控制内存分配,以数据对象(而不是页面)的粒度实现性能分析和数据迁移。该方法弥补了操作系统和应用程序之间的语义鸿沟。通过将数据对象与DNN拓扑关联,Sentinel避免了不必要的数据移动,并主动触发数据移动。仅使用DNN模型峰值内存消耗的20%作为快速内存大小,Sentinel实现了与普通DNN模型上的仅快速内存系统相同或相当的性能(最多8%的性能差异); Sentinel还始终优于最先进的解决方案18%。

英文摘要:

Software-managed heterogeneous memory (HM) provides a promising solution to increase memory capacity and cost efficiency. However, to release the performance potential of HM, we face a problem of data management. Given an application with various execution phases and each with possibly distinct working sets, we must move data between memory components of HM to optimize performance. The deep neural network (DNN), as a common workload on data centers, imposes great challenges on data management on HM. This workload often employs a task dataflow execution model, and is featured with a large amount of small data objects and fine-grained operations (tasks). This execution model imposes challenges on memory profiling and efficient data migration. We present Sentinel, a runtime system that automatically optimizes data migration (i.e., data management) on HM to achieve performance similar to that on the fast memory-only system with a much smaller capacity of fast memory. To achieve this,Sentinel exploits domain knowledge about deep learning to adopt a custom approach for data management. Sentinel leverages workload repeatability to break the dilemma between profiling accuracy and overhead; It enables profiling and data migration at the granularity of data objects (not pages), by controlling memory allocation. This method bridges the semantic gap between operating system and applications. By associating data objects with the DNN topology, Sentinel avoids unnecessary data movement and proactively triggers data movement. Using only 20% of peak memory consumption of DNN models as fast memory size, Sentinel achieves the same or comparable performance (at most 8% performance difference) to that of the fast memory-only system on common DNN models; Sentinel also consistently outperforms a state-of-the-art solution by 18%.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑