LLM中的刻板印象检测：一种多类、可解释且基准驱动的方法
Stereotype Detection in LLMs: A Multiclass, Explainable, and Benchmark-Driven Approach
AI总结：
本文针对LLMs中的刻板印象检测问题，构建了包含51,867个实例的多类MGS数据集，训练并评估了多种规模与架构的微调语言模型分类器，同时结合SHAP、LIME等可解释AI工具验证模型模式，并利用最优分类器对主流LLMs文本生成中的刻板印象进行了基准测试。
摘要：
刻板印象检测是一项具有挑战性且主观的任务，因为某些陈述，例如“黑人喜欢打篮球”，可能看起来没有明显的恶意，但仍然会强化种族刻板印象。随着大型语言模型（LLMs）在面向人类的人工智能（AI）应用中日益普及，检测这些类型的偏见至关重要。然而，LLMs存在延续并放大源自其训练数据的刻板印象输出的风险。一个可靠的刻板印象检测器对于基准测试偏见、监控模型输入和输出、过滤训练数据以及确保下游应用中更公平的模型行为至关重要。本文介绍了多粒度刻板印象（MGS）数据集，包含51,867个实例，涵盖性别、种族、职业、宗教和其他刻板印象，这些实例是从多个现有数据集中精选出来的。我们评估了各种机器学习方法以建立基线，并微调了不同架构和规模的语言模型，提出了一套在MGS数据集上训练的刻板印象多类分类器。鉴于刻板印象的主观性，可解释性对于将模型学习与人类对刻板印象的理解保持一致至关重要。我们采用可解释AI（XAI）工具，包括SHAP、LIME和BertViz，来评估模型学习到的模式是否与人类对刻板印象的直觉一致。此外，我们开发了刻板印象引导提示，并使用表现最佳的刻板印象分类器，对流行LLMs在文本生成任务中存在的刻板印象进行了基准测试。
英文摘要：
Stereotype detection is a challenging and subjective task, as certain statements, such as "Black people like to play basketball," may not appear overtly toxic but still reinforce racial stereotypes. With the increasing prevalence of large language models (LLMs) in human-facing artificial intelligence (AI) applications, detecting these types of biases is essential. However, LLMs risk perpetuating and amplifying stereotypical outputs derived from their training data. A reliable stereotype detector is crucial for benchmarking bias, monitoring model input and output, filtering training data, and ensuring fairer model behavior in downstream applications. This paper introduces the Multi-Grain Stereotype (MGS) dataset, consisting of 51,867 instances across gender, race, profession, religion, and other stereotypes, curated from multiple existing datasets. We evaluate various machine learning approaches to establish baselines and fine-tune language models of different architectures and sizes, presenting a suite of stereotype multiclass classifiers trained on the MGS dataset. Given the subjectivity of stereotypes, explainability is essential to align model learning with human understanding of stereotypes. We employ explainable AI (XAI) tools, including SHAP, LIME, and BertViz, to assess whether the model's learned patterns align with human intuitions about stereotypes.Additionally, we develop stereotype elicitation prompts and benchmark the presence of stereotypes in text generation tasks using popular LLMs, employing the best-performing stereotype classifiers.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
