arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2411.10656cs.SE
收藏

精度与危险:从量化大语言模型中Python代码质量的证明概念

Precision or Peril: A PoC of Python Code Quality from Quantized Large Language Models

Boise State University(博伊西州立大学);University of Tennessee(田纳西大学)

Eric L. Melin, Adam J. Torek, Nasir U. Eisty, Casey Kennington

AI总结:

研究探讨了小型LLM的代码生成性能及量化影响,发现生成代码存在质量和可维护性问题,强调在软件项目中需谨慎验证LLM生成的代码。

摘要:

Context: 大型语言模型(LLM)如GPT-5和LLaMA-405b展现出先进的代码生成能力,但其部署需要大量计算资源和能源。量化可以减少内存足迹和硬件需求,但可能降低代码质量。Objective: 本研究调查了小型LLM的代码生成性能,考察了量化的影响,并通过证明概念(PoC)识别了常见的代码质量问题。Method: 四个开源LLM在Python基准测试中使用代码相似性度量进行评估,分析8位和4位量化效果,并进行静态代码质量评估。Results: 虽然小型LLM可以生成功能性代码,但基准性能受限。量化影响因情况而异,生成的代码存在质量和可维护性问题。Conclusions: 在软件项目中集成LLM生成的代码前,应仔细验证代码质量。

英文摘要:

Context: Large Language Models (LLMs) like GPT-5 and LLaMA-405b exhibit advanced code generation abilities, but their deployment demands substantial computation resources and energy. Quantization can reduce memory footprint and hardware requirements, yet may degrade code quality. Objective: This study investigates code generation performance of smaller LLMs, examines the effect of quantization, and identifies common code quality issues as a proof of concepts (PoC). Method: Four open-source LLMs are evaluated on Python benchmarks using code similarity metrics, with an analysis on 8-bit and 4-bit quantization, alongside static code quality assessment. Results: While smaller LLMs can generate functional code, benchmark performance is limited. Quantization impacts are variable, and generated code exhibits quality and maintainability concerns. Conclusions: LLM-generated code should be carefully validated before integration into software projects.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑