HPDedup：一种用于云中主存储的混合优先级数据恢复机制
HPDedup: A Hybrid Prioritized Data Deduplication Mechanism for Primary Storage in the Cloud
摘要：
消除云主存储中的重复数据可以提高云服务提供商的成本效率，并降低用户使用云服务的成本。现有的主要重复数据删除技术要么使用内联缓存来利用主要工作负载中的局部性，要么使用在系统空闲时间内运行的后处理重复数据删除来避免对I/O性能的负面影响。然而，由于以下两个原因，它们在运行多个服务或应用的云服务器中都不能很好地工作：首先，重复数据写入的时间局部性可能不存在于一些主存储工作负载中，因此内联缓存通常无法实现良好的去重率。其次，后处理重复数据删除允许将重复数据写入磁盘，因此不提供I/O重复数据删除的好处，并且需要高峰值存储容量。本文提出了HPDedup，一种混合优先级数据删除机制，通过融合内联和后处理过程来精确重复数据删除，来处理在位于同一位置的虚拟机或容器中运行的应用程序共享的存储系统。在内联重复数据删除阶段，HPDedup提供了一种指纹缓存机制，该机制估计来自不同VM或应用程序的数据流中重复数据的时间局部性，并根据估计为这些流分配缓存优先级。HPDedup还允许根据流的空间位置为流设置不同的重复数据删除阈值，以减少磁盘碎片。后处理阶段从磁盘中删除由于时间局部性弱而无法缓存指纹的副本。我们的实验结果表明，HPDedup明显优于国家的最先进的主存储重复数据删除技术的内联缓存效率和主要重复数据删除效率。
英文摘要：
Eliminating duplicate data in primary storage of clouds increases the cost-efficiency of cloud service providers as well as reduces the cost of users for using cloud services. Existing primary deduplication techniques either use inline caching to exploit locality in primary workloads or use post-processing deduplication running in system idle time to avoid the negative impact on I/O performance. However, neither of them works well in the cloud servers running multiple services or applications for the following two reasons: Firstly, the temporal locality of duplicate data writes may not exist in some primary storage workloads thus inline caching often fails to achieve good deduplication ratio. Secondly, the post-processing deduplication allows duplicate data to be written into disks, therefore does not provide the benefit of I/O deduplication and requires high peak storage capacity. This paper presents HPDedup, a Hybrid Prioritized data Deduplication mechanism to deal with the storage system shared by applications running in co-located virtual machines or containers by fusing an inline and a post-processing process for exact deduplication. In the inline deduplication phase, HPDedup gives a fingerprint caching mechanism that estimates the temporal locality of duplicates in data streams from different VMs or applications and prioritizes the cache allocation for these streams based on the estimation. HPDedup also allows different deduplication threshold for streams based on their spatial locality to reduce the disk fragmentation. The post-processing phase removes duplicates whose fingerprints are not able to be cached due to the weak temporal locality from disks. Our experimental results show that HPDedup clearly outperforms the state-of-the-art primary storage deduplication techniques in terms of inline cache efficiency and primary deduplication efficiency.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
