用于代码文档生成的大型语言模型比较分析
A Comparative Analysis of Large Language Models for Code Documentation Generation
AI总结：
本文全面比较了GPT-3.5、GPT-4等大型语言模型在代码文档生成中的表现，发现闭源模型在多参数上优于开源模型且超越原始文档，并指出文件级文档生成质量相对较差。
摘要：
本文对大型语言模型（LLMs）在代码文档生成方面进行了全面的比较分析。代码文档是软件编写过程中的重要组成部分。本文在准确性、完整性、相关性、可理解性、可读性和耗时等各种参数上，对GPT-3.5、GPT-4、Bard、Llama2和Starchat等模型在不同级别的代码文档上进行了评估。我们的评估采用基于清单的系统以最小化主观性，从而提供更客观的评估。我们发现，除Starchat外，所有LLMs均持续优于原始文档。值得注意的是，与开源/源码可用的LLMs（即LLama 2和StarChat）相比，闭源模型GPT-3.5、GPT-4和Bard在各项参数上表现出更优越的性能。考虑到生成耗时，GPT-4耗时最长，其次是Llama2和Bard，而ChatGPT和Starchat的生成时间相当。此外，与内联和函数级文档相比，文件级文档在所有参数（耗时除外）上的表现明显较差。
英文摘要：
This paper presents a comprehensive comparative analysis of Large Language Models (LLMs) for generation of code documentation. Code documentation is an essential part of the software writing process. The paper evaluates models such as GPT-3.5, GPT-4, Bard, Llama2, and Starchat on various parameters like Accuracy, Completeness, Relevance, Understandability, Readability and Time Taken for different levels of code documentation. Our evaluation employs a checklist-based system to minimize subjectivity, providing a more objective assessment. We find that, barring Starchat, all LLMs consistently outperform the original documentation. Notably, closed-source models GPT-3.5, GPT-4, and Bard exhibit superior performance across various parameters compared to open-source/source-available LLMs, namely LLama 2 and StarChat. Considering the time taken for generation, GPT-4 demonstrated the longest duration, followed by Llama2, Bard, with ChatGPT and Starchat having comparable generation times. Additionally, file level documentation had a considerably worse performance across all parameters (except for time taken) as compared to inline and function level documentation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
