发表机构
AI Horizon Labs; Universidade Federal do Pampa (UNIPAMPA)（AI地平线实验室; 潘帕联邦大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
该研究针对7款LLMs与SLMs开展Terraform生成基准测试，发现IaC的语法有效性与安全合规性正交，仅靠提示工程不足，自动化多工具扫描是必要补充。
AI 中文摘要
云配置错误仍是安全事件的主要诱因，但大语言模型（LLMs）与小型语言模型（SLMs）能否生成符合安全要求的基础设施即代码（IaC）仍是悬而未决的问题。我们在17种场景下对7款模型进行AWS Terraform生成基准测试，其中包含3款闭源LLMs（Claude Opus 4、GPT-5.4、Gemini 2.5 Pro）和4款开源SLMs（Qwen2.5-Coder-14B、WizardCoder-33B、CodeLlama-13B、Magicoder-S-CL-7B），将Checkov与Trivy扫描工具集成至GitLab CI/CD流水线，在3个安全级别下评估2种提示策略（pass@5）。LLM生成的IaC中，语法有效性与安全合规性基本为正交属性：能可靠生成格式良好Terraform的模型未必能生成安全的Terraform，例如WizardCoder-33B的验证率达77.8%，但Checkov合规率为0；而Claude Opus 4在详细安全提示下达到23.1%的Checkov合规率与92.5%的Trivy通过率。因此，仅靠提示工程是不够的：无论模型家族或提示策略如何，自动化多工具扫描仍是LLM辅助IaC生成的必要补充。所有成果均公开可用。
英文摘要
Cloud misconfiguration remains a leading cause of security incidents, yet whether LLMs and SLMs can generate security-compliant Infrastructure-as-Code is an open question. We benchmark seven models, three closed LLMs (Claude Opus 4, GPT-5.4, Gemini 2.5 Pro) and four open SLMs (Qwen2.5-Coder-14B, WizardCoder-33B, CodeLlama-13B, Magicoder-S-CL-7B), on AWS Terraform generation across 17 scenarios, integrating Checkov and Trivy scanners into a GitLab CI/CD pipeline and evaluating two prompt strategies at three security levels (pass@5). Syntactic validity and security compliance are largely orthogonal properties in LLM-generated IaC, a model that reliably produces well-formed Terraform does not necessarily produce secure Terraform: WizardCoder-33B achieves 77.8% validate rate yet zero Checkov compliance, while Claude Opus 4 reaches 23.1% Checkov and 92.5% Trivy pass rates under detailed security prompting. Consequently, prompt engineering alone is insufficient: automated multi-tool scanning remains a necessary complement to LLM-assisted IaC generation regardless of model family or prompt strategy. All artifacts are publicly available.
Comments10 pages, 1 figure, and 9 tables. The benchmark artifacts and CI/CD pipeline are publicly available at https://gitlab.com/repo-anon-iac/repo-anon-9ac. Accepted for publication at SBSeg 2026
