针对LLM治理的机械执行:在金融决策系统中治理-任务解耦的证据
Mechanical Enforcement for LLM Governance:Evidence of Governance-Task Decoupling in Financial Decision Systems
AI总结:
本文提出五种治理指标,用于评估政策在决策理由层面的合规性,并在合成银行领域比较纯文本治理与机械执行的效果,发现机械执行能显著提升决策信息内容和任务准确性,表明治理与任务评估是两个独立的维度。
摘要:
在受监管的金融工作流程中,大型语言模型由同一模型解释的自然语言政策进行治理,导致主-代理失败:输出可能看似合规但实际不合规。现有评估措施仅关注任务准确性,而未考虑治理在决策理由层面是否约束行为——这是受监管决策必须可审计的地方。我们引入了五个治理指标,用于量化在理由层面的政策合规性,并在合成银行领域应用这些指标,将纯文本治理与机械执行进行比较:四个原始操作在模型的解释循环之外运行。在纯文本治理下,27%的延期不包含与决策相关的信息。机械执行将此比例降低73%,超过翻倍延期信息内容,并将任务准确性从MCC~$0.43提升到$0.88。改进是由架构分离驱动的:在机械执行下,LLM生成的理由与纯文本治理相比具有可比的CDL——收益来自将明确决策从模型的控制中移除。因果消融证实了每个原始操作都是必要的。我们的核心发现是治理-任务解耦:在结构压力下,纯文本治理在两个维度上同时退化,而机械执行在任务性能下降的同时保持治理质量。这表明治理和任务评估是两个独立的轴线:准确性不是受监管AI系统中治理的充分代理。
英文摘要:
Large language models in regulated financial workflows are governed by natural-language policies that the same model interprets, creating a principal--agent failure: outputs can appear compliant without being compliant. Existing evaluation measures task accuracy but not whether governance constrains behaviour at the decision rationale level -- where regulated decisions must be auditable. We introduce five governance metrics that quantify policy compliance at the rationale level and apply them in a synthetic banking domain to compare text-only governance against mechanical enforcement: four primitives operating outside the model's interpretive loop. Under text-only governance, 27% of deferrals carry no decision-relevant information. Mechanical enforcement reduces this rate by 73%, more than doubles deferral information content, and raises task accuracy from MCC~$0.43$ to $0.88$. The improvement is driven by architectural separation: LLM-generated rationales under mechanical enforcement show comparable CDL to text-only governance -- the gain comes from removing clear-cut decisions from the model's control. A causal ablation confirms that each primitive is individually necessary. Our central finding is a governance-task decoupling: under structural stress, text-only governance degrades on both dimensions simultaneously, whereas mechanical enforcement preserves governance quality even as task performance drops. This implies that governance and task evaluation are distinct axes: accuracy is not a sufficient proxy for governance in regulated AI systems.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。