Desbordante：从基准测试套件到高性能科学密集型数据分析器（预印本）
Desbordante: from benchmarking suite to high-performance science-intensive data profiler (preprint)
摘要：
Metanome和OpenClean等开创性的数据分析系统引起了公众对科学密集型数据分析的关注。这种类型的分析旨在提取复杂的模式（原语），如函数依赖关系、数据约束、关联规则等。然而，这些工具是研究原型，而不是生产就绪的系统。下面的工作介绍了Desbordante -一个高性能的科学密集型数据分析器，具有开源代码。与类似的系统不同，它是建立在多用户环境中的工业应用的重点。它是高效的，对崩溃有弹性，并且可扩展。它的效率是通过在C++中实现发现算法来保证的，弹性是通过广泛使用容器化来实现的，可伸缩性是基于容器的复制。Desbordante的目标是向更广泛的公众开放工业级原始发现，专注于非IT专业人员的领域专家。除了发现各种原语之外，Desbordante还提供了原语验证，它不仅报告给定的原语实例是否持有，而且还通过使用特殊屏幕指出是什么阻止了它持有。接下来，Desbordante支持管道-使用发现的原语实现的即用型功能，例如打字错误检测。我们提供了内置的管道，用户可以通过提供的Python绑定构建自己的管道。与其他分析器不同，Desbordante不仅可以处理表格数据，还可以处理图形和事务数据。在本文中，我们介绍了Desbordante，它背后的愿景及其用例。为了提供更深入的视角，我们讨论了它的当前状态、架构和设计决策。此外，我们还概述了我们的未来计划。
英文摘要：
Pioneering data profiling systems such as Metanome and OpenClean brought public attention to science-intensive data profiling. This type of profiling aims to extract complex patterns (primitives) such as functional dependencies, data constraints, association rules, and others. However, these tools are research prototypes rather than production-ready systems. The following work presents Desbordante - a high-performance science-intensive data profiler with open source code. Unlike similar systems, it is built with emphasis on industrial application in a multi-user environment. It is efficient, resilient to crashes, and scalable. Its efficiency is ensured by implementing discovery algorithms in C++, resilience is achieved by extensive use of containerization, and scalability is based on replication of containers. Desbordante aims to open industrial-grade primitive discovery to a broader public, focusing on domain experts who are not IT professionals. Aside from the discovery of various primitives, Desbordante offers primitive validation, which not only reports whether a given instance of primitive holds or not, but also points out what prevents it from holding via the use of special screens. Next, Desbordante supports pipelines - ready-to-use functionality implemented using the discovered primitives, for example, typo detection. We provide built-in pipelines, and the users can construct their own via provided Python bindings. Unlike other profilers, Desbordante works not only with tabular data, but with graph and transactional data as well. In this paper, we present Desbordante, the vision behind it and its use-cases. To provide a more in-depth perspective, we discuss its current state, architecture, and design decisions it is built on. Additionally, we outline our future plans.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
