arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2604.02276cs.AIcs.CLcs.LG

De Jure:基于迭代LLM自优化的结构化监管规则提取

De Jure: Iterative LLM Self-Refinement for Structured Extraction of Regulatory Rules

The Vanguard Group, Inc.(先锋集团)

Keerat Guliani, Deepkamal Gill, David Landsman, Nima Eshraghi, Krishna Kumar, Lovedeep Gondara

AI总结:

De Jure通过四个阶段自动提取结构化监管规则,无需人工标注或领域知识,提升监管文本处理的效率和可追溯性。

摘要:

监管文档编码了具有法律约束力的义务,LLM系统必须尊重这些义务。然而,将密集的、分层结构化的法律文本转换为机器可读规则仍是一个成本高且需要专家参与的过程。我们提出了De Jure,一个完全自动化的、领域无关的管道,用于从原始文档中提取结构化监管规则,不需要人工标注、领域特定的提示或标注的黄金数据。De Jure通过四个连续阶段操作:将源文档标准化为结构化的Markdown;由LLM驱动的语义分解为结构化的规则单元;多标准LLM作为法官评估跨越19个维度的元数据、定义和规则语义;以及在受限制的再生预算内迭代修复低分提取,上游组件在规则单元评估前进行修复。我们在四个模型上评估De Jure,覆盖金融、医疗和AI治理三个监管语料库。在金融领域,De Jure在提取质量上表现出一致且单调的改进,三轮法官指导迭代后达到峰值性能。De Jure在医疗和AI治理领域表现出良好的泛化能力,保持在开放源代码和闭源代码模型上的高性能。在通过RAG的下游合规问题回答评估中,基于De Jure提取规则的响应在单规则检索深度下优于先前工作73.8%的案例,达到更广泛的检索时上升至84.0%,证实了提取的准确性直接转化为下游的实用性。这些结果表明,明确、可解释的评估标准可以替代人类标注,在复杂的监管领域中提供可扩展且可审计的路径,向基于监管的LLM对齐迈进。

英文摘要:

Regulatory documents encode legally binding obligations that LLM-based systems must respect. Yet converting dense, hierarchically structured legal text into machine-readable rules remains a costly, expert-intensive process. We present De Jure, a fully automated, domain-agnostic pipeline for extracting structured regulatory rules from raw documents, requiring no human annotation, domain-specific prompting, or annotated gold data. De Jure operates through four sequential stages: normalization of source documents into structured Markdown; LLM-driven semantic decomposition into structured rule units; multi-criteria LLM-as-a-judge evaluation across 19 dimensions spanning metadata, definitions, and rule semantics; and iterative repair of low-scoring extractions within a bounded regeneration budget, where upstream components are repaired before rule units are evaluated. We evaluate De Jure across four models on three regulatory corpora spanning finance, healthcare, and AI governance. On the finance domain, De Jure yields consistent and monotonic improvement in extraction quality, reaching peak performance within three judge-guided iterations. De Jure generalizes effectively to healthcare and AI governance, maintaining high performance across both open- and closed-source models. In a downstream compliance question-answering evaluation via RAG, responses grounded in De Jure extracted rules are preferred over prior work in 73.8% of cases at single-rule retrieval depth, rising to 84.0% under broader retrieval, confirming that extraction fidelity translates directly into downstream utility. These results demonstrate that explicit, interpretable evaluation criteria can substitute for human annotation in complex regulatory domains, offering a scalable and auditable path toward regulation-grounded LLM alignment.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑