偶尔安全：代码生成助手的比较分析
Ocassionally Secure: A Comparative Analysis of Code Generation Assistants
AI总结：
本文通过 9 项任务和两类开发者角色比较 GPT-3.5、GPT-4、Bard 与 Gemini 的 61 个代码输出，从功能、安全、性能、复杂度和可靠性揭示其适用条件。
摘要：
大语言模型（LLM）正越来越多地用于各类应用，代码生成是一个显著例子。尽管此前研究已经表明 LLM 具备生成安全与不安全代码的能力，但相关文献尚未考虑哪些因素有助于生成安全且有效的代码。因此，本文重点识别并理解 LLM 在真实场景中能够被有效、安全部署以生成高质量代码的条件与情境。我们对四款先进 LLM 进行了比较分析，包括通过 ChatGPT 使用的 GPT-3.5 和 GPT-4，以及来自 Google 的 Bard 和 Gemini；研究使用 9 项独立任务评估每个模型的代码生成能力。我们将研究情境设定为真实开发者在日常工作任务中使用 LLM 的典型用例。此外，我们强调安全意识，这一点通过使用开发者角色的两个不同版本来体现。我们总共收集了 61 个代码输出，并从多个方面进行分析：功能性、安全性、性能、复杂性和可靠性。这些洞察对于理解模型的能力与局限、指导自动化代码生成领域的未来开发和实际应用至关重要。
英文摘要：
$ $Large Language Models (LLMs) are being increasingly utilized in various applications, with code generations being a notable example. While previous research has shown that LLMs have the capability to generate both secure and insecure code, the literature does not take into account what factors help generate secure and effective code. Therefore in this paper we focus on identifying and understanding the conditions and contexts in which LLMs can be effectively and safely deployed in real-world scenarios to generate quality code. We conducted a comparative analysis of four advanced LLMs--GPT-3.5 and GPT-4 using ChatGPT and Bard and Gemini from Google--using 9 separate tasks to assess each model's code generation capabilities. We contextualized our study to represent the typical use cases of a real-life developer employing LLMs for everyday tasks as work. Additionally, we place an emphasis on security awareness which is represented through the use of two distinct versions of our developer persona. In total, we collected 61 code outputs and analyzed them across several aspects: functionality, security, performance, complexity, and reliability. These insights are crucial for understanding the models' capabilities and limitations, guiding future development and practical applications in the field of automated code generation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
