Typical security contests focus on breaking or mitigating the impact of buggy systems. We present the Build-it, Break-it, Fix-it (BIBIFI) contest, which aims to assess the ability to securely build software, not just break it. In BIBIFI, teams build specified software with the goal of maximizing correctness, performance, and security. The latter is tested when teams attempt to break other teams' submissions. Winners are chosen from among the best builders and the best breakers. BIBIFI was designed to be open-ended; teams can use any language, tool, process, etc. that they like. As such, contest outcomes shed light on factors that correlate with successfully building secure software and breaking insecure software. We ran three contests involving a total of 156 teams and three different programming problems. Quantitative analysis from these contests found that the most efficient build-it submissions used C/C++, but submissions coded in a statically-type safe language were 11 times less likely to have a security flaw than C/C++ submissions. Break-it teams that were also successful build-it teams were significantly better at finding security bugs.
Demo: SGCode: A Flexible Prompt-Optimizing System for Secure Generation of Code
New Jersey Institute of Technology(新泽西理工学院);Kent State University(肯特州立大学);Qatar Computing Research Institute(卡塔尔计算研究所);Microsoft Azure AI(微软 Azure 人工智能)
CodeBC: A More Secure Large Language Model for Smart Contract Code Generation in Blockchain
State Key Laboratory of Complex & Critical Software Environment, Beihang University(复杂与关键软件环境国家重点实验室,北京航空航天大学);School of Artificial Intelligence, Beihang University(人工智能学院,北京航空航天大学);Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing(未来区块链与隐私计算北京先进创新中心);Beijing Academy of Blockchain and Edge Computing, BABEC(区块链与边缘计算北京研究院,BABEC)