arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2501.13302cs.CLcs.AI
收藏

观察AI看门狗:AI安全审核分类器的公平性与鲁棒性分析

Watching the AI Watchdogs: A Fairness and Robustness Analysis of AI Safety Moderation Classifiers

King’s Institute for Artificial Intelligence, King’s College London(伦敦国王学院国王人工智能研究所);University of South Florida(南佛罗里达大学)

Akshit Achara, Anshuman Chhabra

AI总结:

本研究分析四种闭源AI安全审核分类器的公平性与鲁棒性,发现存在公平性和鲁棒性差距,需在未来版本中改进。

摘要:

AI安全审核(ASM)分类器旨在审核社交媒体平台上的内容,并作为护栏,防止大型语言模型(LLMs)在不安全输入上进行微调。鉴于其可能产生差异性影响,确保这些分类器:(1)不会不公平地将少数群体用户的内容与多数群体用户相比视为不安全,以及(2)其行为在相似输入下保持鲁棒且一致,至关重要。因此,在本工作中,我们考察了四种广泛使用的闭源ASM分类器的公平性和鲁棒性:OpenAI Moderation API、Perspective API、Google Cloud Natural Language(GCNL)API和Clarifai API。我们使用人口统计均等和条件统计均等等指标评估公平性,并将其性能与ASM模型及仅公平的基线进行比较。此外,我们通过测试分类器对微小和自然输入扰动的敏感性来分析鲁棒性。我们的研究结果揭示了潜在的公平性和鲁棒性差距,强调了在未来版本中缓解这些问题的必要性。

英文摘要:

AI Safety Moderation (ASM) classifiers are designed to moderate content on social media platforms and to serve as guardrails that prevent Large Language Models (LLMs) from being fine-tuned on unsafe inputs. Owing to their potential for disparate impact, it is crucial to ensure that these classifiers: (1) do not unfairly classify content belonging to users from minority groups as unsafe compared to those from majority groups and (2) that their behavior remains robust and consistent across similar inputs. In this work, we thus examine the fairness and robustness of four widely-used, closed-source ASM classifiers: OpenAI Moderation API, Perspective API, Google Cloud Natural Language (GCNL) API, and Clarifai API. We assess fairness using metrics such as demographic parity and conditional statistical parity, comparing their performance against ASM models and a fair-only baseline. Additionally, we analyze robustness by testing the classifiers' sensitivity to small and natural input perturbations. Our findings reveal potential fairness and robustness gaps, highlighting the need to mitigate these issues in future versions of these models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑