大语言模型对非裔美国英语的隐性纠正:通过激活引导来审计和减轻方言偏见
LLMs Silently Correct African American English: Auditing and Mitigating Dialect Bias via Activation Steering
AI总结:
研究发现大语言模型会将非裔美国英语重写为标准美国英语,提出端到端框架审计和减轻偏见,审计用cDGI等方法,减轻偏见用激活引导,该方法比提示更有效,还发布了真实AAE平行语料库。
摘要:
非裔美国英语(AAE)是一种有超过3000万人使用的有规则的方言,却经常被大语言模型(LLMs)误解和“纠正”。研究表明,即使前文是AAE,六个指令微调的LLMs(14B到70B)也系统地偏好标准美国英语(SAE)的延续,将AAE有效地重写成SAE。为此提出了一个端到端框架来审计和减轻这种偏见。审计方面,引入条件方言组不变性(cDGI)和特征级定位分析;减轻偏见方面,首次将激活引导应用于方言偏见,这是一种无训练、测试时的方法。激活引导比提示减少偏见多5到20倍,同时保持SAE流畅性。还发布了迄今为止最大的真实AAE平行语料库REAL-AAE。
英文摘要:
African American English (AAE), a rule-governed dialect spoken by over 30 million people, is routinely misinterpreted and "corrected" by large language models (LLMs). Across six instruction-tuned LLMs (14B to 70B), we show that state-of-the-art models systematically prefer Standard American English (SAE) continuations even when the preceding context is in AAE, effectively rewriting AAE into SAE. We present an end-to-end framework to audit and mitigate this bias. For auditing, we introduce conditional Dialect Group Invariance (cDGI), which isolates true model bias from translator-induced artifacts, and a feature-level localization analysis that identifies which AAE markers most strongly trigger bias; we find that syntactic constructions, especially negative concord (e.g., "ain't nobody"), are universal triggers across all models. For mitigation, we introduce, to our knowledge, the first application of activation steering to dialect bias: a training-free, test-time method that extracts dialect directions via causal tracing and injects them into bias-relevant layers. Activation steering reduces bias 5 to 20 times more than prompting while preserving SAE fluency. To enable this work, we release REAL-AAE , the largest real-AAE parallel corpus to date: 17,479 AAE/SAE/ AAE_back triplets from natural tweets (2 to 6 times larger than prior real-AAE resources), validated automatically (BERTScore F1 = 0.95) and by three native AAE speakers (83.0% semantic agreement).
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。