arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2401.10690cs.LGcs.AIcs.IR
收藏

超越 RMSE 和 MAE:引入 EAUC 揭示二元回归模型中隐藏的偏差与不公平

Beyond RMSE and MAE: Introducing EAUC to unmask hidden bias and unfairness in dyadic regression models

LIDIA Group, CITIC, Universidade da Coruña(LIDIA小组、CITIC、科鲁纳大学)

Jorge Paz-Ruza, Amparo Alonso-Betanzos, Bertha Guijarro-Berdiñas, Brais Cancela, Carlos Eiras-Franco

AI总结:

本文指出二元回归模型存在“偏心偏差”导致关键情况下预测劣于随机,因传统 RMSE 无法捕捉此问题,故提出新指标 EAUC 量化偏差并验证其有效性,为系统公平性提供评估方法。

摘要:

二元回归模型对实体对输出实值预测,在许多领域是基础(例如在 Recommender Systems 中获取用户-产品评分),在其他领域也很有前景且正在探索中(例如在精准药理学中调整患者-药物剂量)。在这项工作中,我们证明了个别实体非均匀的观测值分布会导致 state-of-the-art 模型出现严重偏差,使预测偏向于该实体过去观测值的平均值,并在偏心但关键的情况下提供比随机更差的预测能力;我们将这种现象命名为偏心偏差。我们表明,像 Root Mean Squared Error (RMSE) 这样的全局误差指标不足以捕捉这种偏差,因此我们引入了 Eccentricity-Area Under the Curve (EAUC) 作为一种新指标,能够在所有研究的领域和模型中对其进行量化。我们通过使用朴素的训练后偏差修正进行实验,证明了 EAUC 的直观解释,并理论化了其他使用 EAUC 来指导构建公平模型的选项。这项工作为二元回归提供了一种具备偏差感知的评估方法,以防止此类系统在关键现实应用中产生不公平。

英文摘要:

Dyadic regression models, which output real-valued predictions for pairs of entities, are fundamental in many domains (e.g. obtaining user-product ratings in Recommender Systems) and promising and under exploration in others (e.g. tuning patient-drug dosages in precision pharmacology). In this work, we prove that non-uniform observed value distributions of individual entities lead to severe biases in state-of-the-art models, skewing predictions towards the average of observed past values for the entity and providing worse-than-random predictive power in eccentric yet crucial cases; we name this phenomenon eccentricity bias. We show that global error metrics like Root Mean Squared Error (RMSE) are insufficient to capture this bias, and we introduce Eccentricity-Area Under the Curve (EAUC) as a novel metric that can quantify it in all studied domains and models. We prove the intuitive interpretation of EAUC by experimenting with naive post-training bias corrections, and theorize other options to use EAUC to guide the construction of fair models. This work contributes a bias-aware evaluation of dyadic regression to prevent unfairness in critical real-world applications of such systems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑