arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2407.08188cs.LGcs.CY
收藏

立场:衡量数据集多样性,而非仅声称其具备多样性

Position: Measure Dataset Diversity, Don't Just Claim It

Stanford University(斯坦福大学);Sony AI(索尼人工智能);Technical University of Munich(慕尼黑工业大学)

Dora Zhao, Jerone T. A. Andrews, Orestis Papakyriakopoulos, Alice Xiang

AI总结:

针对机器学习数据集常用的“多样性”等价值负载术语缺乏明确定义与验证的问题,研究分析135个图文数据集,借鉴社会科学测量理论提出数据集多样性的概念化、操作化与评估建议,推动数据集构建更严谨。

摘要:

机器学习(ML)数据集常被视为中立的,但其本质上封装了抽象且有争议的社会建构。数据集构建者频繁使用多样性、偏差、质量这类负载价值的术语来描述数据集。尽管这些术语十分常见,却缺乏清晰的定义与验证。我们的研究通过分析135个图像与文本数据集中的“多样性”,探究了这一问题的影响。我们借鉴社会科学领域的研究,应用测量理论的相关原则,明确了在数据集中概念化、操作化及评估多样性时需考量的要点,并给出了相关建议。我们的研究结果对机器学习研究具有更广泛的意义,倡导在数据集构建过程中采用更细致、更精准的方法处理负载价值的属性。

英文摘要:

Machine learning (ML) datasets, often perceived as neutral, inherently encapsulate abstract and disputed social constructs. Dataset curators frequently employ value-laden terms such as diversity, bias, and quality to characterize datasets. Despite their prevalence, these terms lack clear definitions and validation. Our research explores the implications of this issue by analyzing "diversity" across 135 image and text datasets. Drawing from social sciences, we apply principles from measurement theory to identify considerations and offer recommendations for conceptualizing, operationalizing, and evaluating diversity in datasets. Our findings have broader implications for ML research, advocating for a more nuanced and precise approach to handling value-laden properties in dataset construction.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑