arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2211.09500cs.CY
收藏

一个用于二元分类器技术评估的审计框架

An Audit Framework for Technical Assessment of Binary Classifiers

Debarati Bhaumik, Diptish Dey

摘要:

使用逻辑回归(MLogRM)和随机森林(RFM)的多水平模型越来越多地部署在工业中用于二进制分类的目的。欧盟委员会提出的《人工智能法案》(AIA)要求,在某些条件下,这些模型的应用必须是公平、透明和合乎道德的,因此这意味着对这些模型进行技术评估。本文提出并演示了一个审计框架的RFMs和MLogRM的技术评估,重点放在模型,歧视,透明度和可解释性相关的方面。为了测量这些方面,提出了20个KPI,它们与交通灯风险评估方法配对。使用开源数据集训练RFM和MLogRM模型,计算这些KPI并与交通灯进行比较。流行的可解释性方法,如内核和树SHAP的性能进行评估。该框架预计将协助监管机构对二元分类器进行符合性评估,并使部署此类AI系统的供应商和用户受益,以遵守AIA。

英文摘要:

Multilevel models using logistic regression (MLogRM) and random forest models (RFM) are increasingly deployed in industry for the purpose of binary classification. The European Commission's proposed Artificial Intelligence Act (AIA) necessitates, under certain conditions, that application of such models is fair, transparent, and ethical, which consequently implies technical assessment of these models. This paper proposes and demonstrates an audit framework for technical assessment of RFMs and MLogRMs by focussing on model-, discrimination-, and transparency & explainability-related aspects. To measure these aspects 20 KPIs are proposed, which are paired to a traffic light risk assessment method. An open-source dataset is used to train a RFM and a MLogRM model and these KPIs are computed and compared with the traffic lights. The performance of popular explainability methods such as kernel- and tree-SHAP are assessed. The framework is expected to assist regulatory bodies in performing conformity assessments of binary classifiers and also benefits providers and users deploying such AI-systems to comply with the AIA.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑