大语言模型中的道德敏感性：通过行为剖析和机制可解释性对上下文偏见进行分层评估
Moral Sensitivity in LLMs: A Tiered Evaluation of Contextual Bias via Behavioral Profiling and Mechanistic Interpretability
AI总结：
提出道德敏感性指数（MSI）量化大语言模型在七级压力测试中的偏见概率，并通过行为剖析和机制验证揭示模型偏见随情境变化的U型曲线，发现推理蒸馏会重新激活浅层统计关联。
摘要：
大型语言模型（LLM）越来越多地部署在需要细致伦理推理的环境中，但现有的偏见评估将模型输出简单地视为“有偏见”或“无偏见”。这种二元框架忽略了偏见实际出现的渐进、情境敏感的方式。我们分两个阶段解决这一差距：行为剖析和机制验证。在行为阶段，我们引入了道德敏感性指数（MSI），该指标量化了在从抽象数值问题到基于历史和社会经济不公正场景的七级压力测试中产生偏见输出的概率。评估四个领先模型（Claude 3.5、Qwen 3.5、Llama 3和Gemini 1.5），我们识别出由对齐设计塑造的不同行为特征：例如，Gemini 1.5在社会经济框架下达到第5级时MSI为72.7%，而Claude表现出与基于身份的安全训练一致的强烈抑制。然后，我们在机制上验证这些行为模式。我们选择在所有模型中产生最高MSI分数的犯罪偏见场景作为探针，并将logit透镜、注意力分析、激活修补和语义探针应用于一组受控的六个模型，涵盖三个能力层级：小型语言模型（SLM）、指令微调基础模型和推理蒸馏变体。电路级分析揭示了偏见的U型曲线：SLM表现出强烈的犯罪偏见；扩展到指令微调模型消除了偏见；推理蒸馏将偏见重新引入到类似SLM的水平，尽管参数数量相同，这表明蒸馏以重新激活浅层统计关联的方式压缩了推理轨迹。关键的是，驱动高MSI分数的社会负载线索激活了与机制识别出的相同偏见驱动电路，提供了跨阶段验证。
英文摘要：
Large language models (LLMs) are increasingly deployed in settings that require nuanced ethical reasoning, yet existing bias evaluations treat model outputs as simply "biased" or "unbiased." This binary framing misses the gradual, context-sensitive way bias actually emerges. We address this gap in two stages: behavioral profiling and mechanistic validation. In the behavioral stage, we introduce the Moral Sensitivity Index (MSI), a metric that quantifies the probability of biased output across a graduated, seven-tier stress test ranging from abstract numerical problems to scenarios rooted in historical and socioeconomic injustice. Evaluating four leading models (Claude 3.5, Qwen 3.5, Llama 3, and Gemini 1.5), we identify distinct behavioral signatures shaped by alignment design: for instance, Gemini 1.5 reaches 72.7% MSI by Tier 5 under socioeconomic framing, while Claude exhibits sharp suppression consistent with identity-based safety training. We then verify these behavioral patterns mechanistically. We select criminal-bias scenarios, which produced the highest MSI scores across models, as probes and apply logit lens, attention analysis, activation patching, and semantic probing to a controlled set of six models spanning three capability tiers: small language models (SLMs), instruction-tuned base models, and reasoning-distilled variants. Circuit-level analysis reveals a U-curve of bias: SLMs exhibit strong criminal bias; scaling to instruction-tuned models eliminates it; reasoning distillation reintroduces bias to SLM-like levels despite identical parameter counts, suggesting distillation compresses reasoning traces in ways that reactivate shallow statistical associations. Critically, the socially loaded cues that drive high MSI scores activate the same bias-driving circuits identified mechanistically, providing cross-stage validation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
