arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2311.01490cs.SEcs.AI
收藏

大型语言模型在被提示生成代码解释时的行为

The Behavior of Large Language Models When Prompted to Generate Code Explanations

University of Memphis(孟菲斯大学)

Priti Oli, Rabin Banjade, Jeevan Chapagain, Vasile Rus

AI总结:

本研究系统调查了大型语言模型为入门编程课程代码生成解释的行为,发现解释性质受提示、代码、语言、温度和版本影响,但Java和Python解释的可读性一致,正确性高而完整性、简洁性和特异性较低。

摘要:

本文系统性地研究了大型语言模型(LLMs)为入门编程课程中常见代码示例生成代码解释的情况。我们的研究结果显示,LLMs生成的代码解释在性质上存在显著差异,这些差异受到提示措辞、所考虑的特定代码示例、涉及的编程语言、温度参数以及LLM版本等因素的影响。然而,对于Java和Python,出现了一致的模式,即解释的Flesch-Kincaid可读性水平约为7-8年级,且词汇密度一致,表明有意义词汇占总解释规模的比例。此外,生成的解释在正确性方面始终获得高分,但在其他三个指标上得分较低:完整性、简洁性和特异性。

英文摘要:

This paper systematically investigates the generation of code explanations by Large Language Models (LLMs) for code examples commonly encountered in introductory programming courses. Our findings reveal significant variations in the nature of code explanations produced by LLMs, influenced by factors such as the wording of the prompt, the specific code examples under consideration, the programming language involved, the temperature parameter, and the version of the LLM. However, a consistent pattern emerges for Java and Python, where explanations exhibit a Flesch-Kincaid readability level of approximately 7-8 grade and a consistent lexical density, indicating the proportion of meaningful words relative to the total explanation size. Additionally, the generated explanations consistently achieve high scores for correctness, but lower scores on three other metrics: completeness, conciseness, and specificity.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑