随意对话v2:设计一个大的同意驱动数据集来测量算法偏见和鲁棒性
Casual Conversations v2: Designing a large consent-driven dataset to measure algorithmic bias and robustness
摘要:
开发强大而公平的人工智能系统需要具有全面标签的数据集,以帮助确保相关测量的有效性和合法性。因此,最近的工作重点是收集与人有关的数据集,这些数据集具有精心选择的标签,包括敏感特征,以及使用这些属性进行模型测试和开发的同意书。负责任的数据汇集涉及几个阶段,包括但不限于确定用例场景,选择类别(注释),以便数据适合测量子组的算法偏见,最重要的是确保所选类别/子类别对区域差异具有鲁棒性,并包括尽可能多的子组。Meta继续努力衡量AI算法偏见和鲁棒性(https://ai.facebook.com/blog/shedding-light-on-fairness-in-ai-with-a-new-data-set),正在收集一个大型的同意驱动数据集,其中包含一个全面的类别列表。本文介绍了我们提出的设计,这样的类别和子类别的休闲对话v2。
英文摘要:
Developing robust and fair AI systems require datasets with comprehensive set of labels that can help ensure the validity and legitimacy of relevant measurements. Recent efforts, therefore, focus on collecting person-related datasets that have carefully selected labels, including sensitive characteristics, and consent forms in place to use those attributes for model testing and development. Responsible data collection involves several stages, including but not limited to determining use-case scenarios, selecting categories (annotations) such that the data are fit for the purpose of measuring algorithmic bias for subgroups and most importantly ensure that the selected categories/subcategories are robust to regional diversities and inclusive of as many subgroups as possible. Meta, in a continuation of our efforts to measure AI algorithmic bias and robustness (https://ai.facebook.com/blog/shedding-light-on-fairness-in-ai-with-a-new-data-set), is working on collecting a large consent-driven dataset with a comprehensive list of categories. This paper describes our proposed design of such categories and subcategories for Casual Conversations v2.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。