参与式红队行动的黑暗与光明面：针对刻板印象目标以激发大语言模型有害行为
Dark and Bright Side of Participatory Red-Teaming with Targets of Stereotyping for Eliciting Harmful Behaviors from Large Language Models
AI总结：
本文通过参与式红队行动研究，探讨如何利用刻板印象目标的亲身经历揭示大语言模型的偏见，同时关注参与者心理福祉与赋权。
摘要：
红队行动，即通过对抗性提示来暴露有害行为并评估风险，提供了一种动态方法来揭示大语言模型中潜在的刻板印象偏见。由于这种细微的伤害最好由有亲身经历的人来识别，将刻板印象目标纳入红队成员是至关重要的。然而，在利用他们的亲身经历进行红队行动的同时保护心理福祉仍存在关键挑战。我们对20名因刻板印象而被污名化的韩国非精英大学生进行了实证研究。通过混合方法分析，我们发现参与者将经历的歧视转化为识别偏见的战略专长，同时面临压力和对群体身份的负面反思等心理代价。值得注意的是，红队参与通过其作为AI生态系统的守护者的角色增强了他们的掌控感和赋权感。我们讨论了在设计参与式红队行动时优先考虑对被污名化群体的伦理待遇和赋权的含义。
英文摘要：
Red-teaming, where adversarial prompts are crafted to expose harmful behaviors and assess risks, offers a dynamic approach to surfacing underlying stereotypical bias in large language models. Because such subtle harms are best recognized by those with lived experience, involving targets of stereotyping as red-teamers is essential. However, critical challenges remain in leveraging their lived experience for red-teaming while safeguarding psychological well-being. We conducted an empirical study of participatory red-teaming with 20 individuals stigmatized by stereotypes against nonprestigious college graduates in South Korea. Through mixed methods analysis, we found participants transformed experienced discrimination into strategic expertise for identifying biases, while facing psychological costs such as stress and negative reflections on group identity. Notably, red-team participation enhanced their sense of agency and empowerment through their role as guardians of the AI ecosystem. We discuss implications for designing participatory red-teaming that prioritizes both the ethical treatment and empowerment of stigmatized groups.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
