arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1908.10754cs.SE
收藏

多租户数据平台中数据质量管理的语义模式

A Semantic Schema for Data Quality Management in a Multi-Tenant Data Platform

Ning Zhou, Sandra Garcia Esparza, Lars Marius Garshol

摘要:

Schibsted Media Group是一家全球市场公司,业务遍及20多个国家。它正在进行数字化转型,将数据孤岛转换为基于通用数据平台的多租户系统。基于语义层面上的通用模式的良好数据质量对于在市场上构建成功的数据驱动产品至关重要。为了解决这个问题,我们开发了一个基于语义模式管理系统的数据质量工具,以支持模式的版本控制,测试和转换的演变。它可以监控不同应用程序的数据质量要求,并处理由多个模式版本组成的传入数据。今天,该系统在生产中运行,每天为100多个应用程序处理超过10亿个事件。

英文摘要:

Schibsted Media Group is a global marketplace company with presence in more than 20 countries. It is undergoing a digital transformation to convert data silos to a multi-tenant system based on a common data platform. Good data quality based on a common schema on the semantic level is essential for building successful data-driven products across marketplaces. To solve this challenge, we developed the data quality tooling based on a semantic schema management system to support schema evolution with versioning, testing and transformation. It can monitor the data quality requirements for different applications and handle incoming data consisting of multiple schema versions. Today the system is operating in production and processes over one billion events per day for over 100 applications.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑