表示忠实度:利用自我描述审计算法对人类的决策
Representation Fidelity:Auditing Algorithmic Decisions About Humans Using Self-Descriptions
AI总结:
本文提出通过比较外部输入表示与自我描述之间的差异来评估算法决策的忠实度,并构建了首个评估该指标的基准数据集。
摘要:
本文引入了一个新的维度,用于验证算法对人类的决策,通过测量其表示的忠实度。表示忠实度衡量的是关于一个人的决策是否基于合理的依据。我们通过测量同一人两种表示之间的距离来操作化这一概念:(1) 一个外部规定的输入表示,该表示是决策所基于的;(2) 由人类主体提供的自我描述,仅用于验证输入表示。我们研究了这两种表示之间的差异性质,如何量化这些差异,以及推导出一种通用的表示不匹配类型学,以确定表示忠实度的程度。我们进一步提出了第一个评估表示忠实度的基准,基于贷款发放决策的数据集。我们的贷款发放自我描述语料库2025包含了一个由30000个合成自然语言自我描述组成的大型语料库,这些描述是从德国信用数据集中的申请人对应表示中衍生出来的,同时还包含专家对每对表示之间表示不匹配的注释。
英文摘要:
This paper introduces a new dimension for validating algorithmic decisions about humans by measuring the fidelity of their representations. Representation Fidelity measures if decisions about a person rest on reasonable grounds. We propose to operationalize this notion by measuring the distance between two representations of the same person: (1) an externally prescribed input representation on which the decision is based, and (2) a self-description provided by the human subject of the decision, used solely to validate the input representation. We examine the nature of discrepancies between these representations, how such discrepancies can be quantified, and derive a generic typology of representation mismatches that determine the degree of representation fidelity. We further present the first benchmark for evaluating representation fidelity based on a dataset of loan-granting decisions. Our Loan-Granting Self-Representations Corpus 2025 consists of a large corpus of 30 000 synthetic natural language self-descriptions derived from corresponding representations of applicants in the German Credit Dataset, along with expert annotations of representation mismatches between each pair of representations.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。