发表机构
Oracle AI（甲骨文人工智能）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
该研究测试53个模型在11个数据集的四类安全场景下的表现，发现前沿模型在部分场景落后于专用模型，实际对话安全未解决，挑战规模即安全的假设并提供模型选择框架。
AI 中文摘要
大型语言模型（LLMs）正越来越多地被部署到实际应用中，但它们仍然容易生成有害内容。从绕过安全过滤器的对抗性越狱到难以检测的隐性仇恨，这些模型带来的风险范围持续扩大。虽然专用内容审核模型和通用大型语言模型都被用作安全层，但哪种模型最适合哪种类型的有害内容这一问题仍未得到解答。我们开展了迄今为止最全面的大型语言模型安全能力评估，系统测试了53个模型，涵盖11个数据集，我们将这些数据集分为四个不同类别。我们在仅提示和提示-响应两种设置下进行的评估揭示了关键盲区：在某一类别表现领先的大型前沿模型，在其他类别上却显著落后于规模更小的专用替代模型，且所有模型家族在实际对话安全方面基本未得到解决。这些发现挑战了仅靠规模就能确保安全的假设，并为研究界提供了用于明智选择模型的结构化框架。
英文摘要
Large Language Models (LLMs) are increasingly deployed in real-world applications, yet they remain vulnerable to generating harmful content. From adversarial jailbreaks that bypass safety filters to implicit hate that evades detection, the range of risks these models pose continues to grow. While both specialized content moderators and general-purpose LLMs are being used as safety layers, the question of which model is best suited for which type of harmful content remains unanswered. We present the most comprehensive evaluation of LLM safety capabilities to date, systematically testing \textbf{53} models across \textbf{11} datasets that we organize into four distinct categories. Our evaluation under both prompt-only and prompt-response settings uncovers critical blind spots: large frontier models that lead on one category fall significantly behind smaller, specialized alternatives on others, and real-world conversational safety remains largely unsolved across all model families. These findings challenge the assumption that scale alone ensures safety, and provide the community with a structured framework for informed model selection.
CommentsEMNLP 2026 Main Track
