arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2604.21090cs.SEcs.AI
收藏

实践者AI治理提示的结构质量缺口:基于五原则评估框架的实证研究

Structural Quality Gaps in Practitioner AI Governance Prompts: An Empirical Study Using a Five-Principle Evaluation Framework

Nuphirho Research(Nuphirho研究)

Christo Zietsman

AI总结:

研究通过五原则框架评估34个AGENTS.md治理文件,发现37%的文件模型对数据分类和评估标准缺失,揭示了AI治理提示的结构不完整问题及改进方向。

摘要:

AI治理计划日益依赖自然语言提示来约束和指导AI代理行为。这些提示作为可执行规范:它们定义了代理的使命、范围和质量标准。尽管如此,目前尚无系统框架用于评估治理提示是否结构上完整。我们引入了基于计算理论、证明论和贝叶斯知识论的五原则评估框架,并将其应用于34个公开可用的AGENTS.md治理文件。我们的评估发现,37%的评估文件-模型对得分低于结构完整性阈值,其中数据分类和评估标准最常缺失。这些结果表明,实践者编写的治理提示表现出一致的结构模式,自动化静态分析可以检测并修复这些问题。我们讨论了AI辅助开发环境中的需求工程实践影响,识别了AGENTS.md约定中先前未记录的 Artefact 分类缺口,并提出了工具支持的方向。

英文摘要:

AI governance programmes increasingly rely on natural language prompts to constrain and direct AI agent behaviour. These prompts function as executable specifications: they define the agent's mandate, scope, and quality criteria. Despite this role, no systematic framework exists for evaluating whether a governance prompt is structurally complete. We introduce a five-principle evaluation framework grounded in computability theory, proof theory, and Bayesian epistemology, and apply it to an empirical corpus of 34 publicly available AGENTS.md governance files sourced from GitHub. Our evaluation reveals that 37% of evaluated file-model pairs score below the structural completeness threshold, with data classification and assessment rubric criteria most frequently absent. These results suggest that practitioner-authored governance prompts exhibit consistent structural patterns that automated static analysis could detect and remediate. We discuss implications for requirements engineering practice in AI-assisted development contexts, identify a previously undocumented artefact classification gap in the AGENTS.md convention, and propose directions for tool support.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑