arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2601.09762cs.SEcs.AI
收藏

从LLMs中阐释隐性监管知识以实现需求的自动形式化,用于合规测试用例生成

Explicating Tacit Regulatory Knowledge from LLMs to Auto-Formalize Requirements for Compliance Test Case Generation

Shanghai Key Laboratory of Trustworthy Computing, ECNU(上海可信计算实验室,华东师范大学);Dishui Lake International Software Engineering Institute, ECNU(迪希湖国际软件工程研究院,华东师范大学)

Zhiyi Xue, Xiaohong Chen, Min Zhang

AI总结:

RAFT通过从多个LLMs中阐释隐性监管知识,实现需求自动形式化和合规测试生成,提升测试用例生成的精度和效率。

摘要:

在高度监管的领域中,合规测试至关重要但大多依赖人工,需要领域专家将复杂的法规转换为可执行的测试用例。尽管大型语言模型(LLMs)在自动化方面显示出潜力,但其容易产生幻觉限制了可靠的应用。现有的混合方法通过用形式模型限制LLMs来缓解这一问题,但仍然依赖于昂贵的手动建模。为了解决这个问题,本文提出RAFT,一个通过从多个LLMs中阐释隐性监管知识来实现需求自动形式化和合规测试生成的框架。RAFT采用自适应净化-聚合策略,从多个LLMs中阐释隐性监管知识,并将其整合到三个艺术制品中:一个领域元模型、一个形式化的需求表示和可测试性约束。这些艺术制品随后被动态注入提示中,以指导高精度需求形式化和自动化测试生成。在金融、汽车和电力领域中的实验表明,RAFT实现了专家水平的性能,显著优于最先进的(SOTA)方法,同时减少了整体生成和审查时间。

英文摘要:

Compliance testing in highly regulated domains is crucial but largely manual, requiring domain experts to translate complex regulations into executable test cases. While large language models (LLMs) show promise for automation, their susceptibility to hallucinations limits reliable application. Existing hybrid approaches mitigate this issue by constraining LLMs with formal models, but still rely on costly manual modeling. To solve this problem, this paper proposes RAFT, a framework for requirements auto-formalization and compliance test generation via explicating tacit regulatory knowledge from multiple LLMs. RAFT employs an Adaptive Purification-Aggregation strategy to explicate tacit regulatory knowledge from multiple LLMs and integrate it into three artifacts: a domain meta-model, a formal requirements representation, and testability constraints. These artifacts are then dynamically injected into prompts to guide high-precision requirement formalization and automated test generation. Experiments across financial, automotive, and power domains show that RAFT achieves expert-level performance, substantially outperforms state-of-the-art (SOTA) methods while reducing overall generation and review time.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑