arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2503.02115cs.DB
收藏

通用数据协调框架:支持RADx数据中心中可复现且可扩展的数据集成

A General-Purpose Data Harmonization Framework: Supporting Reproducible and Scalable Data Integration in the RADx Data Hub

Jimmy K. Yu, Marcos Martínez-Romero, Matthew Horridge, Mete U. Akdogan, Mark A. Musen

AI总结:

提出一个通用数据协调框架,通过可参数化基本操作构建转换并自动化记录,支持RADx数据中心的透明可复现数据集成。

摘要:

在大数据时代,初级研究数据遵循FAIR原则(可查找性、可访问性、互操作性和可重用性)至关重要。数据协调通过将异构数据对齐到标准化表示下,增强了互操作性和可重用性,既惠及负责维护数据质量标准的存储库管理员,也惠及需要统一数据集的消费者。然而,数据协调在实践中颇具挑战,需要大量的领域和技术专业知识。我们提出了一个软件框架,以促进有原则且可复现的协调协议。我们的框架实现了一种新颖的策略,即从可参数化的基本操作(例如将数值分配给用户指定的类别)构建协调转换,并对已执行的转换进行自动化记录。我们建立了数据表示模型和协调策略,然后报告了在RADx数据中心背景下的概念验证应用。我们的框架使数据从业者能够执行透明且可复现的协调协议,这些协议与其研究目标紧密对齐。

英文摘要:

In the age of big data, it is important for primary research data to follow the FAIR principles of findability, accessibility, interoperability, and reusability. Data harmonization enhances interoperability and reusability by aligning heterogeneous data under standardized representations, benefiting both repository curators responsible for upholding data quality standards and consumers who require unified datasets. However, data harmonization is difficult in practice, requiring significant domain and technical expertise. We present a software framework to facilitate principled and reproducible harmonization protocols. Our framework implements a novel strategy of building harmonization transformations from parameterizable primitive operations, such as the assignment of numerical values to user-specified categories, with automated bookkeeping for executed transformations. We establish our data representation model and harmonization strategy and then report a proof-of-concept application in the context of the RADx Data Hub. Our framework enables data practitioners to execute transparent and reproducible harmonization protocols that align closely with their research goals.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑