在AUDAL数据湖中对文本文档和表格数据进行联合管理和分析
Joint Management and Analysis of Textual Documents and Tabular Data within the AUDAL Data Lake
摘要:
2010年,数据湖的概念作为数据仓库的替代方案出现,用于大数据管理。数据湖遵循schema-on-read方法,提供丰富而灵活的分析。然而,尽管在工业界和学术界都很流行,但数据湖的概念仍在成熟,数据湖设计的方法论方法仍然很少。因此,我们引入了一种新的方法来设计一个数据湖,并提出了一个广泛的元数据系统,以激活更丰富的功能比通常支持的数据湖方法。我们在AUDAL数据湖中实现了我们的方法,在那里我们联合利用文本文档和表格数据,与文献中通常在数据湖中处理的结构化和/或半结构化数据相比。此外,我们还通过利用元数据来激活数据检索和内容分析(包括Text-OLAP和SQL查询)进行创新。最后,我们展示了我们的方法的可行性,一方面使用真实的用例,另一方面使用基准测试。
英文摘要:
In 2010, the concept of data lake emerged as an alternative to data warehouses for big data management. Data lakes follow a schema-on-read approach to provide rich and flexible analyses. However, although trendy in both the industry and academia, the concept of data lake is still maturing, and there are still few methodological approaches to data lake design. Thus, we introduce a new approach to design a data lake and propose an extensive metadata system to activate richer features than those usually supported in data lake approaches. We implement our approach in the AUDAL data lake, where we jointly exploit both textual documents and tabular data, in contrast with structured and/or semi-structured data typically processed in data lakes from the literature. Furthermore, we also innovate by leveraging metadata to activate both data retrieval and content analysis, including Text-OLAP and SQL querying. Finally, we show the feasibility of our approach using a real-word use case on the one hand, and a benchmark on the other hand.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。