arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2510.24958cs.CY
收藏

自适应数据收集用于拉美社区源的刻板印象评估(LACES)

Adaptive Data Collection for Latin-American Community-sourced Evaluation of Stereotypes (LACES)

Universidad Nacional de Córdoba(科尔多瓦国立大学);Fundación Vía Libre(Vía Libre基金会);CONICET(阿根廷国家科学研究委员会);Google Research(谷歌研究)

Guido Ivetta, Pietro Palombini, Sofía Martinelli, Marcos J Gomez, M. María Echeveste, Sunipa Dev, Vinodkumar Prabhakaran, Luciana Benotti

AI总结:

本文提出LACES数据集和自适应数据收集方法,用于评估拉丁美洲地区的刻板印象,以弥补现有研究的地理文化缺口。

摘要:

对NLP模型中的社会偏见进行评估受到地理文化差距的严重阻碍,这使得像拉美地区这样的区域严重被忽视,使得无法充分评估或减轻语言技术中有害的地域刻板印象的持续传播。本文提出了LACES,一个针对15个拉丁美洲国家的刻板印象关联数据集。该数据集包含4789个刻板印象关联,由83名参与者手动创建和标注。该数据集通过在拉丁美洲进行有针对性的社区合作开发而成。此外,本文提出了一种新的自适应数据收集方法,该方法独特地将新刻板印象条目的获取与现有数据的验证整合到一个统一的工作流程中。这种方法产生了一个包含更多独特刻板印象的资源,从而实现了更高效的刻板印象收集。本文还通过展示在该数据集上去偏方法的效果比现有流行的刻板印象基准测试更差,来支持LACES的质量。

英文摘要:

The evaluation of societal biases in NLP models is critically hindered by a geo-cultural gap, This leaves regions such as Latin America severely underserved, making it impossible to adequately assess or mitigate the perpetuation of harmful regional stereotypes in language technologies. This paper presents LACES, a stereotype association dataset, for 15 Latin American countries. This dataset includes 4,789 stereotype associations manually created and annotated by 83 participants. The dataset was developed through targeted community partnerships across Latin America. Additionally, in this paper, we propose a novel adaptive data collection methodology that uniquely integrates the sourcing of new stereotype entries and the validation of existing data within a single, unified workflow. This approach results in a resource with more unique stereotypes than previous static collection methods, enabling a more efficient stereotype collection. The paper further supports the quality of LACES by demonstrating reduced efficacy of debiasing methods on this dataset in comparison to existing popular stereotype benchmarks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑