研究平衡、过滤和复杂性对预测多重性的影响:以数据为中心的视角
Investigating the Impact of Balancing, Filtering, and Complexity on Predictive Multiplicity: A Data-Centric Perspective
AI总结:
本文从以数据为中心的视角,在21个真实数据集上评估平衡与过滤预处理及数据复杂性对预测多重性和模型稳定性的影响。
摘要:
Rashomon效应给模型选择带来了重大挑战。当多个模型在数据集上取得相似性能却产生不同预测时,就会出现这种效应,从而导致预测多重性。这在高风险环境中尤其成问题,因为任意的模型结果可能造成严重后果。传统模型选择方法优先考虑准确性,未能解决这一问题。类别不平衡和无关变量等因素进一步使情况复杂化,使模型更难提供可信的预测。以数据为中心的AI方法可以通过优先进行数据优化来缓解这些问题,尤其是借助预处理技术。然而,近期研究表明,预处理方法可能会无意中加剧预测多重性。本文在考虑数据复杂性的情况下,研究平衡和过滤方法等数据预处理技术如何影响预测多重性和模型稳定性。我们在21个真实世界数据集上开展实验,应用多种平衡和过滤技术,并利用Rashomon效应评估这些方法引入的预测多重性水平。此外,我们还考察过滤技术如何减少冗余并增强模型泛化能力。研究结果揭示了平衡方法、数据复杂性与预测多重性之间的关系,展示了以数据为中心的AI策略如何改善模型性能。
英文摘要:
The Rashomon effect presents a significant challenge in model selection. It occurs when multiple models achieve similar performance on a dataset but produce different predictions, resulting in predictive multiplicity. This is especially problematic in high-stakes environments, where arbitrary model outcomes can have serious consequences. Traditional model selection methods prioritize accuracy and fail to address this issue. Factors such as class imbalance and irrelevant variables further complicate the situation, making it harder for models to provide trustworthy predictions. Data-centric AI approaches can mitigate these problems by prioritizing data optimization, particularly through preprocessing techniques. However, recent studies suggest preprocessing methods may inadvertently inflate predictive multiplicity. This paper investigates how data preprocessing techniques like balancing and filtering methods impact predictive multiplicity and model stability, considering the complexity of the data. We conduct the experiments on 21 real-world datasets, applying various balancing and filtering techniques, and assess the level of predictive multiplicity introduced by these methods by leveraging the Rashomon effect. Additionally, we examine how filtering techniques reduce redundancy and enhance model generalization. The findings provide insights into the relationship between balancing methods, data complexity, and predictive multiplicity, demonstrating how data-centric AI strategies can improve model performance.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。