人群规模药物流行病学数据集的交互式探索
Interactive exploration of population scale pharmacoepidemiology datasets
摘要:
与药物不良反应(ADR)数据相关的人口规模药物处方数据支持足够大的模型的拟合,以检测使用传统方法在较小数据集上无法检测的药物使用和ADR模式。然而,在大型数据集中检测ADR模式需要用于可扩展数据处理的工具、用于数据分析的机器学习和交互式可视化。据我们所知,没有现有的药物流行病学工具支持所有三个要求。因此,我们创建了一个工具,用于交互式探索数百万样本的处方数据集中的模式。我们使用Spark对数据进行预处理,以便进行机器学习和使用SQL查询进行分析。我们已经在Keras和scikit-learn框架中实现了模型。模型结果是可视化的,并使用实时Python编码在Python中进行解释。我们应用我们的工具来探索来自挪威处方数据库的3.84亿个处方数据集,以及6200万个住院老年人的处方。我们在两分钟内预处理数据,在几秒钟内训练模型,并以毫秒为单位绘制结果。我们的研究结果显示了结合计算能力,计算时间短,易于使用的人口规模的药物流行病学数据集分析的力量。该代码是开源的,可在https://github.com/uit-hdl/norpd_prescription_analyses上获得
英文摘要:
Population-scale drug prescription data linked with adverse drug reaction (ADR) data supports the fitting of models large enough to detect drug use and ADR patterns that are not detectable using traditional methods on smaller datasets. However, detecting ADR patterns in large datasets requires tools for scalable data processing, machine learning for data analysis, and interactive visualization. To our knowledge no existing pharmacoepidemiology tool supports all three requirements. We have therefore created a tool for interactive exploration of patterns in prescription datasets with millions of samples. We use Spark to preprocess the data for machine learning and for analyses using SQL queries. We have implemented models in Keras and the scikit-learn framework. The model results are visualized and interpreted using live Python coding in Jupyter. We apply our tool to explore a 384 million prescription data set from the Norwegian Prescription Database combined with a 62 million prescriptions for elders that were hospitalized. We preprocess the data in two minutes, train models in seconds, and plot the results in milliseconds. Our results show the power of combining computational power, short computation times, and ease of use for analysis of population scale pharmacoepidemiology datasets. The code is open source and available at: https://github.com/uit-hdl/norpd_prescription_analyses
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。