数据湖仓一体:数据仓库及其更多
The Data Lakehouse: Data Warehousing and More
AI总结:
本文提出数据湖仓一体架构,融合RDBMS-OLAP与云数据湖优势,解决传统数据仓库的局限,并分解数据仓库组件以展示其满足性及额外能力。
摘要:
为在线分析处理(OLAP)设计的关系数据库管理系统(RDBMS-OLAP)多年来一直是数据民主化以及实现商业智能和报告等分析用例的基础。然而,RDBMS-OLAP系统存在一些众所周知的挑战。它们主要仅针对关系型工作负载进行优化,导致数据副本激增,可能变得难以管理,并且由于数据以专有格式存储,可能导致供应商锁定,限制了对引擎、工具以及供应商所提供能力之外功能的访问。随着数据驱动决策需求的激增,对更强大的数据架构来解决这些挑战的需求变得日益关键。云数据湖解决了RDBMS-OLAP系统的一些缺点,但它们自身也带来了一系列挑战。最近,组织通常采用两层架构方法,以利用这两个平台的优势,同时利用云数据湖和RDBMS-OLAP系统。然而,这种方法带来了额外的挑战、复杂性和开销。本文讨论了数据湖仓一体(一种新的架构方法)如何实现RDBMS-OLAP和云数据湖相结合所带来的相同好处,同时还提供了额外的优势。我们将当今的数据仓库分解为与实现无关的组件、能力和实践。然后,我们利用这些方面来展示湖仓一体架构如何满足它们。接着,我们更进一步,讨论湖仓一体架构相对于RDBMS-OLAP提供了哪些额外的能力和优势。
英文摘要:
Relational Database Management Systems designed for Online Analytical Processing (RDBMS-OLAP) have been foundational to democratizing data and enabling analytical use cases such as business intelligence and reporting for many years. However, RDBMS-OLAP systems present some well-known challenges. They are primarily optimized only for relational workloads, lead to proliferation of data copies which can become unmanageable, and since the data is stored in proprietary formats, it can lead to vendor lock-in, restricting access to engines, tools, and capabilities beyond what the vendor offers. As the demand for data-driven decision making surges, the need for a more robust data architecture to address these challenges becomes ever more critical. Cloud data lakes have addressed some of the shortcomings of RDBMS-OLAP systems, but they present their own set of challenges. More recently, organizations have often followed a two-tier architectural approach to take advantage of both these platforms, leveraging both cloud data lakes and RDBMS-OLAP systems. However, this approach brings additional challenges, complexities, and overhead. This paper discusses how a data lakehouse, a new architectural approach, achieves the same benefits of an RDBMS-OLAP and cloud data lake combined, while also providing additional advantages. We take today's data warehousing and break it down into implementation independent components, capabilities, and practices. We then take these aspects and show how a lakehouse architecture satisfies them. Then, we go a step further and discuss what additional capabilities and benefits a lakehouse architecture provides over an RDBMS-OLAP.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。