vibe编码是未来吗？对LLM生成代码用于建设安全的实证评估
Is Vibe Coding the Future? An Empirical Assessment of LLM Generated Codes for Construction Safety
- Department of Construction Management, Colorado State University（科罗拉多州立大学建设管理系）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
研究评估了450个由三个前沿模型生成的Python脚本在建设安全中的可靠性、软件架构和领域特定安全精度，发现用户角色与数据幻觉存在显著关系，且存在高达45%的静默故障率。
AI中文摘要：
vibe编码作为一种非技术用户通过自然语言指导大语言模型（LLM）生成可执行代码的范式，为建筑行业带来了机遇和风险。本研究通过评估450个由Claude 3.5 Haiku、GPT-4o-Mini和Gemini 2.5 Flash生成的Python脚本，发现LLM生成的代码在建设安全领域存在严重局限。研究发现用户角色与数据幻觉之间存在显著关系，且静默故障率高达45%，其中GPT-4o-Mini在功能性代码中存在56%的数学不准确输出。结果表明，当前LLM缺乏用于独立安全工程所需的确定性严谨性，需要采用确定性AI包装器和严格治理以确保物理- cyber部署的安全性。
英文摘要：
The emergence of vibe coding, a paradigm where non-technical users instruct Large Language Models (LLMs) to generate executable codes via natural language, presents both significant opportunities and severe risks for the construction industry. While empowering construction personnel such as the safety managers, foremen, and workers to develop tools and software, the probabilistic nature of LLMs introduces the threat of silent failures, wherein generated code compiles perfectly but executes flawed mathematical safety logic. This study empirically evaluates the reliability, software architecture, and domain-specific safety fidelity of 450 vibe-coded Python scripts generated by three frontier models, Claude 3.5 Haiku, GPT-4o-Mini, and Gemini 2.5 Flash. Utilizing a persona-driven prompt dataset (n=150) and a bifurcated evaluation pipeline comprising isolated dynamic sandboxing and an LLM-as-a-Judge, the research quantifies the severe limits of zero-shot vibe codes for construction safety. The findings reveal a highly significant relationship between user persona and data hallucination, demonstrating that less formal prompts drastically increase the AI's propensity to invent missing safety variables. Furthermore, while the models demonstrated high foundational execution viability (~85%), this syntactic reliability actively masked logic deficits and a severe lack of defensive programming. Among successfully executed scripts, the study identified an alarming ~45% overall Silent Failure Rate, with GPT-4o-Mini generating mathematically inaccurate outputs in ~56% of its functional code. The results demonstrate that current LLMs lack the deterministic rigor required for standalone safety engineering, necessitating the adoption of deterministic AI wrappers and strict governance for cyber-physical deployments.
