Construction remains one of the most hazardous sectors. Recent advancements in AI, particularly Large Language Models (LLMs), offer promising opportunities for enhancing workplace safety. However, responsible integration of LLMs requires systematic evaluation, as deploying them without understanding their capabilities and limitations risks generating inaccurate information, fostering misplaced confidence, and compromising worker safety. This study evaluates the performance of two widely used LLMs, GPT-3.5 and GPT-4o, across three standardized exams administered by the Board of Certified Safety Professionals (BCSP). Using 385 questions spanning seven safety knowledge areas, the study analyzes the models' accuracy, consistency, and reliability. Results show that both models consistently exceed the BCSP benchmark, with GPT-4o achieving an accuracy rate of 84.6% and GPT-3.5 reaching 73.8%. Both models demonstrate strengths in safety management systems and hazard identification and control, but exhibit weaknesses in science, mathematics, emergency response, and fire prevention. An error analysis identifies four primary limitations affecting LLM performance: lack of knowledge, reasoning flaws, memory issues, and calculation errors. Our study also highlights the impact of prompt engineering strategies, with variations in accuracy reaching 13.5% for GPT-3.5 and 7.9% for GPT-4o. However, no single prompt configuration proves universally effective. This research advances knowledge in three ways: by identifying areas where LLMs can support safety practices and where human oversight remains essential, by offering practical insights into improving LLM implementation through prompt engineering, and by providing evidence-based direction for future research and development. These contributions support the responsible integration of AI in construction safety management toward achieving zero injuries.
When Do Model Internals Help? Exploring the Role of Representation Engineering in LLM Safety
State Key Laboratory of Multimedia Information Processing, Peking University(北京大学多媒体信息处理国家重点实验室);School of Computer Science, Peking University(北京大学计算机科学学院)
Prompt Engineering Strategies for LLM-based Qualitative Coding of Psychological Safety in Software Engineering Communities: A Controlled Empirical Study
Federal University of Bahia(巴伊亚联邦大学);State University of Feira de Santana(费拉德桑塔纳州立大学);Federal University of Sergipe(塞格皮联邦大学);Federal Institute of Ceara(塞阿拉联邦理工学院)
Safety Alignment Should Be Made More Than Just A Few Attention Heads
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络安全学院)
Towards Understanding Safety Alignment: A Mechanistic Perspective from Safety Neurons
Department of Computer Science and Technology, BNRist(计算机科学与技术系,BNRist);Shenzhen International Graduate School(深圳国际研究生院);KIRC, Institute for Artificial Intelligence, Tsinghua University(人工智能研究院,清华大学)
Prompt Engineering for Responsible Generative AI Use in African Education: A Report from a Three-Day Training Series
Mampong Technical College of Education(曼庞教育技术学院);Northern Cape Provincial Department of Education(北开普省教育部);Bagabaga College of Education(巴加巴加教育学院);Gambaga College of Education(甘巴加教育学院);St. Joseph’s College of Education(圣约瑟夫教育学院);Federal University of Transportation, Daura(道拉联邦交通大学);University of Education, Winneba(温尼巴教育大学);University of Manitoba(曼尼托巴大学);University for Development Studies(发展研究大学);JET Education Services(JET教育服务机构);Generative AI for Education and Research in Africa (GenAI-ERA)(非洲生成式人工智能教育与研究组织(GenAI-ERA));National University of Lesotho(莱索托国立大学);University of Georgia(佐治亚大学)
On AI Safety and Security Technical Debt in Engineering AI-Enabled Systems
University of Birmingham(伯明翰大学);HBKU(哈马德·本·哈利法大学);Bournemouth University(伯恩茅斯大学);Brunel University London(伦敦布鲁内尔大学);University of Hawaii(夏威夷大学);College of Science and Engineering, HBKU(哈马德·本·哈利法大学科学与工程学院)
RADIANT-LLM: an Agentic Retrieval Augmented Generation Framework for Reliable Decision Support in Safety-Critical Nuclear Engineering
Department of Nuclear Engineering, Texas A\&M University, College Station, TX, USA;College of Performance, Visualization;Fine Arts, Texas A\&M University, College Station, TX, USA
Are We Aligned? A Preliminary Investigation of the Alignment of Responsible AI Values between LLMs and Human Judgment
Information and Computer Science Department, KFUPM(信息与计算机科学系,KFUPM);IRC for finance and digital economy, KFUPM(金融与数字经济研究中心,KFUPM);SDAIA-KFUPM Joint Research Center for Artificial Intelligence, KFUPM(人工智能联合研究中心,KFUPM)
Secure Deep Learning Engineering: A Software Quality Assurance Perspective
Harbin Institute of Technology(哈尔滨工业大学);Nanyang Technological University(南洋理工大学);Carnegie Mellon University(卡内基梅隆大学);Optus Macquarie University Cyber Security Hub(Optus麦考瑞大学网络安全中心);Kyushu University(九州大学);University of Illinois at Urbana–Champaign(伊利诺伊大学厄巴纳-香槟分校);NVIDIA AI Tech Centre(英伟达人工智能技术中心)