瑞士基准003：评估大语言模型在瑞士监管环境中的可靠性与对抗安全性
Swiss-Bench 003: Evaluating LLM Reliability and Adversarial Security for Swiss Regulatory Contexts
AI总结：
本文提出瑞士基准003，扩展HAAS评估框架至八个维度，评估十种前沿模型在瑞士特定任务中的可靠性与对抗安全性，揭示模型在数据保护和安全方面的表现差异。
摘要：
在瑞士金融和监管环境中部署大语言模型（LLMs）需要实证证据证明其生产可靠性和对抗安全性，这两个维度在现有瑞士聚焦评估框架中未被联合操作化。本文介绍了瑞士基准003（SBP-003），通过增加D7（自我评分可靠性代理）和D8（对抗安全性）将HAAS（Helvetic AI Assessment Score）从六个维度扩展至八个维度。我评估了十种前沿模型在808个瑞士特定项目中的表现，涵盖四种语言（德语、法语、意大利语、英语）的七个瑞士适应性基准（瑞士TruthfulQA、瑞士IFEval、瑞士SimpleQA、瑞士NIAH、瑞士PII-Scope、系统提示泄露、瑞士德语理解），目标是FINMA指导08/2024、修订的联邦数据保护法（nDSG）和OWASP Top 10针对LLM。自我评分的D7分数（73-94%）远高于外部评分的D8安全分数（20-61%），尽管这些维度使用不可比的评分制度。系统提示泄露抵抗率从24.8%到88.2%，而PII提取防御在所有模型中仍较弱（14-42%）。Qwen 3.5 Plus在自我评分D7分数上最高（94.4%），而GPT-oss 120B在D8分数上最高（60.7%）尽管是评估中成本最低的模型。所有评估均在提供方默认设置下进行零样本；D7是自我评分，不构成独立验证的准确性。我提供了将基准维度与FINMA模型验证要求、nDSG数据保护义务和OWASP LLM风险类别相关联的概念映射表。
英文摘要：
The deployment of large language models (LLMs) in Swiss financial and regulatory contexts demands empirical evidence of both production reliability and adversarial security, dimensions not jointly operationalized in existing Swiss-focused evaluation frameworks. This paper introduces Swiss-Bench 003 (SBP-003), extending the HAAS (Helvetic AI Assessment Score) from six to eight dimensions by adding D7 (Self-Graded Reliability Proxy) and D8 (Adversarial Security). I evaluate ten frontier models across 808 Swiss-specific items in four languages (German, French, Italian, English), comprising seven Swiss-adapted benchmarks (Swiss TruthfulQA, Swiss IFEval, Swiss SimpleQA, Swiss NIAH, Swiss PII-Scope, System Prompt Leakage, and Swiss German Comprehension) targeting FINMA Guidance 08/2024, the revised Federal Act on Data Protection (nDSG), and OWASP Top 10 for LLMs. Self-graded D7 scores (73-94%) exceed externally judged D8 security scores (20-61%) by a wide margin, though these dimensions use non-comparable scoring regimes. System prompt leakage resistance ranges from 24.8% to 88.2%, while PII extraction defense remains weak (14-42%) across all models. Qwen 3.5 Plus achieves the highest self-graded D7 score (94.4%), while GPT-oss 120B achieves the highest D8 score (60.7%) despite being the lowest-cost model evaluated. All evaluations are zero-shot under provider default settings; D7 is self-graded and does not constitute independently validated accuracy. I provide conceptual mapping tables relating benchmark dimensions to FINMA model validation requirements, nDSG data protection obligations, and OWASP LLM risk categories.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
