基于开源技术的健康数据协作云计算框架
Collaborative Cloud Computing Framework for Health Data with Open Source Technologies
摘要：
传感器技术的扩散和数据汇集方法的进步使得能够积累非常大量的数据。这些数据集越来越多地被考虑用于科学研究。然而，系统体系结构的设计要在并行化、查询处理时间、异构数据类型（例如，时间序列、图像、结构化数据等），以及复制科学研究的困难仍然是一个重大挑战。这对于健康科学研究尤其如此，其中系统必须i）易于使用，具有在最细粒度级别操作数据的灵活性，ii）与程序设计语言内核无关，iii）可扩展，以及iv）符合HIPAA隐私法。在本文中，我们回顾了现有的文献，这种大数据系统在健康科学的科学研究，并确定当前的系统景观的差距。我们提出了一种新的架构，用于软件-硬件-数据生态系统，在分布式环境中使用开源技术，如Apache Hadoop，Kubernetes和ApacheyterHub。我们还使用6900万例患者的大型临床数据集对该系统进行了评估。
英文摘要：
The proliferation of sensor technologies and advancements in data collection methods have enabled the accumulation of very large amounts of data. Increasingly, these datasets are considered for scientific research. However, the design of the system architecture to achieve high performance in terms of parallelization, query processing time, aggregation of heterogeneous data types (e.g., time series, images, structured data, among others), and difficulty in reproducing scientific research remain a major challenge. This is specifically true for health sciences research, where the systems must be i) easy to use with the flexibility to manipulate data at the most granular level, ii) agnostic of programming language kernel, iii) scalable, and iv) compliant with the HIPAA privacy law. In this paper, we review the existing literature for such big data systems for scientific research in health sciences and identify the gaps of the current system landscape. We propose a novel architecture for software-hardware-data ecosystem using open source technologies such as Apache Hadoop, Kubernetes and JupyterHub in a distributed environment. We also evaluate the system using a large clinical data set of 69M patients.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
