CSSBench: 评估轻量级大语言模型对中文特定对抗模式的安全性
CSSBench: Evaluating the Safety of Lightweight LLMs against Chinese-Specific Adversarial Patterns
AI总结：
CSSBench通过评估中文特定对抗模式，揭示轻量级大语言模型在中文环境下的安全挑战，为实际应用提供安全评估框架。
摘要：
大型语言模型（LLMs）越来越多地部署在成本敏感和设备端场景中，而安全护栏主要在英语领域取得了进展。然而，现实中的中文恶意查询通常通过同音字、拼音、符号分割等中文特定模式来隐藏意图。这些中文特定对抗模式造成了一个安全评估缺口，现有专注于英语的基准无法很好地捕捉到这一点。这个缺口尤其令人担忧，因为轻量级模型可能更容易受到此类特定对抗扰动的影响。为弥合这一缺口，我们引入了中文特定安全基准（CSSBench），该基准强调这些对抗模式，并评估轻量级LLMs在中文环境中的安全性。我们的基准覆盖了六个常见的中文场景领域，包括非法活动与合规、隐私泄露、健康和医疗误导信息、欺诈与仇恨、成人内容以及公共与政治安全，并将查询组织成多种任务类型。我们评估了一组流行的轻量级LLMs，并测量过度拒绝行为以评估安全导致的性能下降。我们的结果表明，中文特定对抗模式是轻量级LLMs的关键挑战。该基准提供了对中文环境下LLMs安全性的全面评估，有助于实际中的稳健部署。
英文摘要：
Large language models (LLMs) are increasingly deployed in cost-sensitive and on-device scenarios, and safety guardrails have advanced mainly in English. However, real-world Chinese malicious queries typically conceal intent via homophones, pinyin, symbol-based splitting, and other Chinese-specific patterns. These Chinese-specific adversarial patterns create the safety evaluation gap that is not well captured by existing benchmarks focused on English. This gap is particularly concerning for lightweight models, which may be more vulnerable to such specific adversarial perturbations. To bridge this gap, we introduce the Chinese-Specific Safety Benchmark (CSSBench) that emphasizes these adversarial patterns and evaluates the safety of lightweight LLMs in Chinese. Our benchmark covers six domains that are common in real Chinese scenarios, including illegal activities and compliance, privacy leakage, health and medical misinformation, fraud and hate, adult content, and public and political safety, and organizes queries into multiple task types. We evaluate a set of popular lightweight LLMs and measure over-refusal behavior to assess safety-induced performance degradation. Our results show that the Chinese-specific adversarial pattern is a critical challenge for lightweight LLMs. This benchmark offers a comprehensive evaluation of LLM safety in Chinese, assisting robust deployments in practice.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
