arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2607.08028cs.AIcs.CLcs.SE
收藏

从提示到契约:用于可审计企业大语言模型代理的 harness 工程

From Prompts to Contracts: Harness Engineering for Auditable Enterprise LLM Agents

AI Leadership Research Center(人工智能领导力研究中心)

Joongho Ahn, Moonsoo Kim

AI总结:

研究针对企业大语言模型应用从原型到产品化的需求,提出 harness 工程方法,在韩国企业数据切片上实例化并评估三个问题,包括契约保留、模型替换检查及代码执行效果,形成可将原型转为可审计应用的工程模式。

摘要:

企业大语言模型应用通常始于由提示和检索上下文驱动行为的原型。产品化增加了对源边界、实体路由、答案契约和可重现跟踪的要求。我们提出一种 harness 工程方法,将此模式重构为可跟踪、可审计的大语言模型代理架构:确定性行为转移到代码、清单、模式和围绕可替换组合边界的验证工件中,而源支持的声明仍是运行时答案的权威。我们在五个韩国企业集团(25 家上市公司)的公共数据切片上实例化它,并评估三个研究问题。一是 harness 在固定验证场景中保留其源基础、实体路由、跟踪、输出卫生和推荐语言契约;二是检查 harness 在模型替换下的执行情况,跨三个托管模型,在所有 270 次组合边界运行中通过,失败局限于模型组合端并被捕获记录;三是代码拥有的保证是有承载能力的,仅靠提示无法重现,仅改变执行层,提示指令会让推荐语言和内部跟踪泄漏违规到达读者,而 harness 完全阻止,外部护栏虽能防止违规但过度拒绝降低效用,只有代码拥有的执行能同时保留安全和效用。结果是一种可重复使用的工程模式,用于将探索性原型转变为具有版本化源、控制和验证工件的可审计应用程序。

英文摘要:

Enterprise large language model (LLM) applications often begin as prototypes whose behavior is carried by prompts and retrieval context. Productization adds requirements for source boundaries, entity routing, answer contracts, and reproducible traces. We present a harness-engineering approach that reconstructs this pattern into a traceable, auditable LLM-agent architecture: deterministic behavior moves into code, manifests, schemas, and validation artifacts around a replaceable composition boundary, while source-backed claims remain the authority for runtime answers. We instantiate it on a public-data slice of five Korean corporate groups (25 listed companies) and evaluate three research questions. (1) The harness preserves its source-grounding, entity-routing, trace, output-hygiene, and recommendation-language contracts across the fixed validation scenarios; a fault-injection control confirms the validators flag deliberately broken contracts. (2) The checks the harness enforces held under model substitution: across three hosted models, they passed on all 270 composition-boundary runs; failures were confined to the model-composed side and were caught and recorded. (3) The code-owned guarantees are load-bearing, not reproducible by prompting alone: holding the model fixed and varying only the enforcement layer, prompt instructions alone let recommendation-language and internal-trace-leakage violations reach the reader, which the harness blocks entirely. A bolt-on external guardrail prevents such violations too but over-refuses, dropping utility to 88/120 where the harness preserves full utility (120/120); in this ablation, only code-owned enforcement preserves both safety and utility. The result is a reusable engineering pattern for turning exploratory prototypes into auditable applications with versioned source, control, and validation artifacts.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑