arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2501.04835cs.SEcs.AI
收藏

代码大语言模型理解设计模式吗?

Do Code LLMs Understand Design Patterns?

Northwestern University(西北大学);Zhejiang University(浙江大学);Alibaba Group(阿里巴巴集团)

Zhenyu Pan, Xuefeng Song, Yunkun Wang, Rongyu Cao, Binhua Li, Yongbin Li, Han Liu

AI总结:

本文通过识别、理解与生成三类实验,实证评估 Code LLMs 对软件设计模式的掌握情况,揭示其固有偏差会显著降低下游任务可靠性。

摘要:

代码大语言模型(Code Large Language Models,Code LLMs)在适应代码生成与补全、缺陷检测与修复等各类下游任务方面表现出很强的通用性。然而,Code LLMs 往往无法把握既有编码规范,导致生成的代码与给定项目所要求的设计模式相冲突。因此,开发者必须进行后处理,使生成代码符合项目的设计规范。在本研究中,我们通过实证方式考察 Code LLMs 在软件开发中的偏差。通过精心设计的实验,我们从识别、理解和生成三个方面评估模型对设计模式的理解。研究结果表明,Code LLMs 中的偏差会显著影响下游任务的可靠性。

英文摘要:

Code Large Language Models (LLMs) demonstrate great versatility in adapting to various downstream tasks, including code generation and completion, as well as bug detection and fixing. However, Code LLMs often fail to capture existing coding standards, leading to the generation of code that conflicts with the required design patterns for a given project. As a result, developers must post-process to adapt the generated code to the project's design norms. In this work, we empirically investigate the biases of Code LLMs in software development. Through carefully designed experiments, we assess the models' understanding of design patterns across recognition, comprehension, and generation. Our findings reveal that biases in Code LLMs significantly affect the reliability of downstream tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑