Brame:面向云边端协同的分层数据管理框架
Brame: Hierarchical Data Management Framework for Cloud-Edge-Device Collaboration
AI总结:
针对云边端协同数据管理缺乏系统理论方法及现有分层管理存在的运维成本高、数据定位难等问题,提出基于块的分层数据管理框架Brame,通过工作负载感知三层架构及离线块生成方法提升性能。
摘要:
在大数据领域,云边端协同在工业场景中十分普遍。然而,该领域缺乏对数据管理相关理论和方法的系统性探索。本文深入研究云边端协同环境下的数据存储与调度子问题。通过对云边协同数据管理的特征和需求进行广泛研究与分析,现有分层数据管理研究主要聚焦于冷热数据迁移。此外,这些研究面临运维成本高、分层存储中数据定位困难,以及因管理粒度过细导致的元数据管理复杂等挑战,这些挑战阻碍了云边端协同存储需求的满足。为克服这些挑战,我们提出一种基于块的分层数据管理框架Brame,该框架倡导工作负载感知的三层存储架构,并建议将数据管理的基本单元从元组转为块。Brame具备离线块生成方法,旨在实现高效的块生成和快速的查询路由。大量实验证实了Brame的优越性能。
英文摘要:
In the realm of big data, cloud-edge-device collaboration is prevalent in industrial scenarios. However, a systematic exploration of the theory and methodologies related to data management in this field is lacking. This paper delves into the sub-problem of data storage and scheduling within cloud-edge-device collaborative environments. Following extensive research and analysis of the characteristics and requirements of data management in cloud-edge collaboration, it is evident that existing studies on hierarchical data management primarily focus on the migration of hot and cold data. Additionally, these studies encounter challenges such as elevated operational and maintenance costs, difficulties in locating data within tiered storage, and intricate metadata management attributable to excessively fine-grained management granularity. These challenges impede the fulfillment of the storage needs in cloud-edge-device collaboration. To overcome these challenges, we propose a \underline{B}lock-based hie\underline{R}archical d\underline{A}ta \underline{M}anagement fram\underline{E}work, \textbf{Brame}, which advocates for a workload-aware three-tier storage architecture and suggests a shift from using tuples to employing $Blocks$ as the fundamental unit for data management. \textbf{Brame} owns an offline block generation method designed to facilitate efficient block generation and expeditious query routing. Extensive experiments substantiate the superior performance of \textbf{Brame}.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。