机器学习流程中数据泄露对跨站点类别不平衡数据协调的影响
Impact of Leakage on Data Harmonization in Machine Learning Pipelines in Class Imbalance Across Sites
AI总结:
本研究针对跨站点类别不平衡的机器学习数据协调中数据泄露问题,提出PrettYharmonize方法,通过假装目标标签协调数据,在避免泄露的同时达到可比性能。
摘要:
机器学习(ML)模型受益于大型数据集。在生物医学领域收集数据成本高昂且具有挑战性,因此,合并数据集已成为常见做法。然而,在不同条件下获得的数据集可能呈现出不期望的站点特异性变异。数据协调方法旨在去除站点特异性变异,同时保留生物学相关信息。本研究评估了在跨站点类别不平衡的场景下,广泛使用的基于ComBat的方法在数据协调中的有效性。我们发现这些方法存在数据泄露问题。为解决此问题,我们提出了一种新方法PrettYharmonize,通过假装目标标签来协调数据。我们使用为基准测试协调效用而设计的受控数据集验证了我们的方法。最后,使用真实世界的MRI和临床数据,我们将易泄露的方法与PrettYharmonize进行比较,并表明其在避免数据泄露的同时实现了可比的性能,特别是在站点-目标依赖场景中。
英文摘要:
Machine learning (ML) models benefit from large datasets. Collecting data in biomedical domains is costly and challenging, hence, combining datasets has become a common practice. However, datasets obtained under different conditions could present undesired site-specific variability. Data harmonization methods aim to remove site-specific variance while retaining biologically relevant information. This study evaluates the effectiveness of popularly used ComBat-based methods for harmonizing data in scenarios where the class balance is not equal across sites. We find that these methods struggle with data leakage issues. To overcome this problem, we propose a novel approach PrettYharmonize, designed to harmonize data by pretending the target labels. We validate our approach using controlled datasets designed to benchmark the utility of harmonization. Finally, using real-world MRI and clinical data, we compare leakage-prone methods with PrettYharmonize and show that it achieves comparable performance while avoiding data leakage, particularly in site-target-dependence scenarios.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。