arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2401.16340cs.SE
收藏

库版本在开发者与ChatGPT对话中的作用

The role of library versions in Developer-ChatGPT conversations

Rachna Raj, Diego Elias Costa

AI总结:

本研究通过分析含4000多条交互的DevGPT数据集,发现开发者与ChatGPT的代码对话中仅9%提及库版本约束,且多由用户提出,同时定性识别出相关潜在问题。

摘要:

大语言模型(LLM)的最新突破为ChatGPT等软件开发工具赋能,以辅助开发者完成复杂任务。开发者使用ChatGPT编写代码、审查代码变更,甚至调试程序。在这些交互中,ChatGPT经常推荐依赖外部库的代码片段。然而,库的代码会随时间演变,导致曾经正确的代码片段失效,使得推荐代码难以复用。\n在本研究中,我们分析了DevGPT(一个包含4000多条开发者与ChatGPT交互记录的数据集),以了解库版本在代码相关对话中的作用。我们量化了代码相关对话中提及库版本约束的频率,以及ChatGPT推荐安装特定库的时机。研究结果表明,尽管ChatGPT不断推荐和分析带有外部依赖的代码,但库版本约束仅出现在9%的对话中。在大多数对话中,版本约束由用户提出(而非由ChatGPT指定),作为获取更高质量回复的一种方式。此外,我们通过定性方法研究了对话中库版本约束的使用方式,识别出若干值得进一步研究的潜在问题。

英文摘要:

The latest breakthroughs in large language models (LLM) have empowered software development tools, such as ChatGPT, to aid developers in complex tasks. Developers use ChatGPT to write code, review code changes, and even debug their programs. In these interactions, ChatGPT often recommends code snippets that depend on external libraries. However, code from libraries changes over time, invalidating a once-correct code snippet and making it difficult to reuse recommended code. In this study, we analyze DevGPT, a dataset of more than 4,000 Developer-ChatGPT interactions, to understand the role of library versions in code-related conversations. We quantify how often library version constraints are mentioned in code-related conversations and when ChatGPT recommends the installation of specific libraries. Our findings show that, albeit to constantly recommend and analyze code with external dependencies, library version constraints only appear in 9% of the conversations. In the majority of conversations, the version constraints are prompted by users (as opposed to being specified by ChatGPT) as a method for receiving better quality responses. Moreover, we study how library version constraints are used in the conversation through qualitative methods, identifying several potential problems that warrant further research.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑