可审计且可复用的交叉映射表,用于快速、规模化整合分散的表格数据
Auditable and reusable crosswalks for fast, scaled integration of scattered tabular data
AI总结:
本文提出一个开源策展工具包,通过模式到模式的高层交叉映射脚本,实现分散表格数据的可审计、可复用和规模化整合,并提供 Python 包与无代码可视化 Web 应用。
摘要:
本文介绍了一个开源策展工具包,旨在生成结构良好且可互操作的数据。策展过程被划分为离散组件,以模式(schema)为中心,对复杂且分散的表格数据进行可审计的重构,使其符合目标模式。任务分离使得在源数据不存在的情况下也能开发软件和分析。转换过程被捕获为描述模式到模式映射的高层顺序脚本,从而降低复杂性和资源需求。最终数据被转换,但目标是任何满足模式定义的数据都可以通过交叉映射表(crosswalk)进行重构。该工具包既可作为 Python 包使用,也可作为“无代码”可视化 Web 应用使用。文中展示了一个可视化示例,该示例源自一项纵向研究,其中来自数百个地方议会的分散源数据被整合到一个单一数据库中。
英文摘要:
This paper presents an open-source curatorial toolkit intended to produce well-structured and interoperable data. Curation is divided into discrete components, with a schema-centric focus for auditable restructuring of complex and scattered tabular data to conform to a destination schema. Task separation allows development of software and analysis without source data being present. Transformations are captured as high-level sequential scripts describing schema-to-schema mappings, reducing complexity and resource requirements. Ultimately, data are transformed, but the objective is that any data meeting a schema definition can be restructured using a crosswalk. The toolkit is available both as a Python package, and as a 'no-code' visual web application. A visual example is presented, derived from a longitudinal study where scattered source data from hundreds of local councils are integrated into a single database.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。