arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.04837cs.AI
收藏

设计行为代码（DBCs）：一种以分类为导向的分层治理基准用于大型语言模型

Design Behaviour Codes (DBCs): A Taxonomy-Driven Layered Governance Benchmark for Large Language Models

Yonih Ventures（约尼赫风险投资公司）；Ramaiah University of Applied Sciences（拉迈亚应用科学大学）

G. Madan Mohan, Veena Kiran Nambiar, Kiranmayee Janardhan

备注：
14 pages, 3 figures
阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出DBC基准，用于评估大型语言模型中的行为治理层，通过三臂实验展示DBC在降低风险暴露率和提升合规性方面的效果。

摘要：

我们引入了动态行为约束（DBC）基准，这是首个评估结构化、150控制行为治理层（MDBC）有效性的经验框架，该治理层在大型语言模型（LLMs）的推理过程中应用。不同于训练时间对齐方法（RLHF、DPO）或事后内容审查API，DBC构成一个系统提示级别的治理层，具有模型无关、管辖可映射和可审计的特性。我们通过一个代理红队协议，使用五个对抗性攻击策略（直接、角色扮演、少样本、假设、权威冒充）在3个模型家族中，评估DBC框架在30个领域风险分类（分为六个集群：幻觉与校准、偏见与公平性、恶意使用、隐私与数据保护、鲁棒性与可靠性、对齐代理）中的表现。我们的三臂受控设计（基础、基础加审查、基础加DBC）能够因果地归因于风险降低。关键发现：DBC层将总体风险暴露率（RER）从7.19%（基础）降至4.55%（基础加DBC），代表36.8%的相对风险降低，相比标准安全审查提示的0.6%。MDBC符合度评分从8.6（基础）提升至8.7（基础加DBC）。欧盟人工智能法案合规性（自动评分）在DBC层达到8.5。三法官评估集合产生Fleiss kappa大于0.70（实质性一致），验证了我们的自动化流程。集群消融识别出完整性保护集群（MDBC 081 099）提供了最高的每个领域风险降低，而灰盒对抗攻击实现了4.83%的DBC绕过率。我们发布了基准代码、提示数据库和所有评估成果，以实现可重复性和长期跟踪，随着模型的发展。

英文摘要：

We introduce the Dynamic Behavioral Constraint (DBC) benchmark, the first empirical framework for evaluating the efficacy of a structured, 150-control behavioral governance layer, the MDBC (Madan DBC) system, applied at inference time to large language models (LLMs). Unlike training time alignment methods (RLHF, DPO) or post-hoc content moderation APIs, DBCs constitute a system prompt level governance layer that is model-agnostic, jurisdiction-mappable, and auditable. We evaluate the DBC Framework across a 30 domain risk taxonomy organized into six clusters (Hallucination and Calibration, Bias and Fairness, Malicious Use, Privacy and Data Protection, Robustness and Reliability, and Misalignment Agency) using an agentic red-team protocol with five adversarial attack strategies (Direct, Roleplay, Few-Shot, Hypothetical, Authority Spoof) across 3 model families. Our three-arm controlled design (Base, Base plus Moderation, Base plus DBC) enables causal attribution of risk reduction. Key findings: the DBC layer reduces the aggregate Risk Exposure Rate (RER) from 7.19 percent (Base) to 4.55 percent (Base plus DBC), representing a 36.8 percent relative risk reduction, compared with 0.6 percent for a standard safety moderation prompt. MDBC Adherence Scores improve from 8.6 by 10 (Base) to 8.7 by 10 (Base plus DBC). EU AI Act compliance (automated scoring) reaches 8.5by 10 under the DBC layer. A three judge evaluation ensemble yields Fleiss kappa greater than 0.70 (substantial agreement), validating our automated pipeline. Cluster ablation identifies the Integrity Protection cluster (MDBC 081 099) as delivering the highest per domain risk reduction, while graybox adversarial attacks achieve a DBC Bypass Rate of 4.83 percent . We release the benchmark code, prompt database, and all evaluation artefacts to enable reproducibility and longitudinal tracking as models evolve.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑