arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2305.09402cs.SEcs.CL
收藏

GPT-3.5与Bard AI模型在Java函数代码生成能力上的初步分析

A Preliminary Analysis on the Code Generation Capabilities of GPT-3.5 and Bard AI Models for Java Functions

Brunel University London(伦敦布鲁内尔大学);University College London(伦敦大学学院);University of Cagliari(卡利亚里大学)

Giuseppe Destefanis, Silvia Bartolucci, Marco Ortu

AI总结:

本文评估了GPT-3.5和Bard在Java代码生成方面的能力,发现GPT-3.5在CodingBat测试中以90.6%的正确率显著优于Bard的53.1%,揭示了AI辅助代码生成工具的改进方向。

摘要:

本文评估了两种最先进的人工智能(AI)模型——GPT-3.5和Bard在给定函数描述下生成Java代码的能力。我们从CodingBat.com获取了描述,这是一个提供编程练习题的流行在线平台。我们基于正确性比较了两个模型生成的Java代码,该正确性通过平台自身的测试用例进行验证。结果表明两个模型的能力存在明显差异。GPT-3.5表现出更优的性能,为约90.6%的函数描述生成了正确的代码,而Bard为53.1%的函数生成了正确的代码。尽管两个模型都表现出优势和不足,但这些发现为开发和改进更先进的AI辅助代码生成工具提供了潜在途径。该研究强调了AI在自动化和支持软件开发方面的潜力,尽管需要进一步研究以充分实现这一潜力。

英文摘要:

This paper evaluates the capability of two state-of-the-art artificial intelligence (AI) models, GPT-3.5 and Bard, in generating Java code given a function description. We sourced the descriptions from CodingBat.com, a popular online platform that provides practice problems to learn programming. We compared the Java code generated by both models based on correctness, verified through the platform's own test cases. The results indicate clear differences in the capabilities of the two models. GPT-3.5 demonstrated superior performance, generating correct code for approximately 90.6% of the function descriptions, whereas Bard produced correct code for 53.1% of the functions. While both models exhibited strengths and weaknesses, these findings suggest potential avenues for the development and refinement of more advanced AI-assisted code generation tools. The study underlines the potential of AI in automating and supporting aspects of software development, although further research is required to fully realize this potential.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑