大语言模型在安全领域是否反映了人口多样性？
Do Large Language Models Reflect Demographic Pluralism in Safety?
AI总结：
Demo-SafetyBench通过在提示级别建模人口多样性，实现了在安全评估中兼顾可扩展性和人口鲁棒性。
摘要：
大语言模型（LLM）的安全性本质上是多元化的，反映了道德规范、文化期望和人口背景的变化。然而，现有的对齐数据集，如ANTHROPIC-HH和DICES，依赖于人口狭窄的标注者群体，忽略了不同社区在安全感知上的差异。Demo-SafetyBench通过在提示级别直接建模人口多样性，将价值框架与响应解耦。在第一阶段，使用Mistral 7B-Instruct-v0.3对DICES的提示重新分类为14个安全领域（改编自BEAVERTAILS），保留人口元数据并通过Llama-3.1-8B-Instruct进行SimHash基于去重，扩展低资源领域，得到43,050个样本。在第二阶段，使用LLM-as-Raters-Gemma-7B、GPT-4o和LLaMA-2-7B在零样本推理下评估多元敏感性。平衡阈值（delta = 0.5，tau = 10）实现了高可靠性（ICC = 0.87）和低人口敏感性（DS = 0.12），证实了多元安全评估既可以扩展又可以人口鲁棒。
英文摘要：
Large Language Model (LLM) safety is inherently pluralistic, reflecting variations in moral norms, cultural expectations, and demographic contexts. Yet, existing alignment datasets such as ANTHROPIC-HH and DICES rely on demographically narrow annotator pools, overlooking variation in safety perception across communities. Demo-SafetyBench addresses this gap by modeling demographic pluralism directly at the prompt level, decoupling value framing from responses. In Stage I, prompts from DICES are reclassified into 14 safety domains (adapted from BEAVERTAILS) using Mistral 7B-Instruct-v0.3, retaining demographic metadata and expanding low-resource domains via Llama-3.1-8B-Instruct with SimHash-based deduplication, yielding 43,050 samples. In Stage II, pluralistic sensitivity is evaluated using LLMs-as-Raters-Gemma-7B, GPT-4o, and LLaMA-2-7B-under zero-shot inference. Balanced thresholds (delta = 0.5, tau = 10) achieve high reliability (ICC = 0.87) and low demographic sensitivity (DS = 0.12), confirming that pluralistic safety evaluation can be both scalable and demographically robust.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
