背景:由大型语言模型(LLMs)驱动的 AI 代码生成正在彻底改变软件开发。OpenAI 的 Codex 和 GPT-4 等模型,以及 DeepSeek,利用大量的代码和自然语言数据集。然而,确保代码质量、正确性以及管理复杂任务仍然具有挑战性,因此需要进行彻底的评估。方法:本研究使用在线判题编码挑战,比较了 ChatGPT(o1 版本)和 DeepSeek(R1 版本)在 Python 代码生成方面的表现。评估内容包括正确性(在线判题结果,最多尝试三次)、代码质量(Pylint/Flake8)和效率(执行时间/内存使用)。结果:DeepSeek 表现出更高的正确性,尤其是在算法任务上,通常能在第一次尝试时就获得“通过”。ChatGPT 有时需要多次尝试或失败。ChatGPT 遇到的问题较少,内存使用相当或略少,执行时间更短,编写的代码行数也更少。结论:DeepSeek 在 Python 代码生成方面表现出更优的正确性,通常需要更少的尝试次数,这表明其在算法问题解决方面具有优势。两个模型在执行时间和内存使用方面的效率几乎相同。最后,本研究为开发人员选择 AI 编码助手提供了见解,并为未来 AI 驱动的软件开发研究提供了信息。
英文摘要:
Background: AI-powered code generation, fueled by Large Language Models (LLMs), is revolutionizing software development. Models like OpenAI's Codex and GPT-4, alongside DeepSeek, leverage vast code and natural language datasets. However, ensuring code quality, correctness, and managing complex tasks remains challenging, necessitating thorough evaluation. Methodology: This research compares ChatGPT (version o1) and DeepSeek (version R1) for Python code generation using online judge coding challenges. It evaluates correctness (online judge verdicts, up to three attempts), code quality (Pylint/Flake8), and efficiency (execution time/memory usage). Results: DeepSeek demonstrated higher correctness, particularly on algorithmic tasks, often achieving 'Accepted' on the first attempt. ChatGPT sometimes requires multiple attempts or failures. ChatGPT encountered fewer issues, used comparable or slightly less memory, consumed less execution times and wrote fewer lines of code. Conclusion: DeepSeek exhibited superior correctness in Python code generation, often requiring fewer attempts, suggesting an advantage in algorithmic problem-solving. Both models showed almost similar efficiency in execution time and memory use. Finally, this research provides insights for developers choosing AI coding assistants and informs future AI-driven software development research.
Can AI be a Teaching Partner? Evaluating ChatGPT, Gemini, and DeepSeek across Three Teaching Strategies
Federal Institute of São Paulo (IFSP)(圣保罗联邦学院);Graduate School of Education of University of Pennsylvania (UPENN)(宾夕法尼亚大学教育研究生院);Institute of Mathematics and Computer Sciences of University of São Paulo (USP)(圣保罗大学数学与计算机科学研究所);Nucleus of Excellence in Social Technologies (NEES) of Federal University of Alagoas (UFAL)(阿拉戈斯联邦大学社会技术卓越中心)
An Insight into Security Code Review with LLMs: Capabilities, Obstacles, and Influential Factors
School of Computer Science, Wuhan University, China(武汉大学计算机科学学院);School of Mathematical and Computational Sciences, Massey University(梅西大学数学与计算科学学院);School of Computing Technologies, RMIT University, Australia(皇家墨尔本理工学院计算技术学院)
A DeepSeek-Powered AI System for Automated Chest Radiograph Interpretation in Clinical Practice
Wuhan University(武汉大学);Union Hospital, Tongji Medical College, Huazhong University of Science and Technology(华中科技大学同济医学院附属协和医院);Hubei Provincial Clinical Research Center for Precision Radiology & Interventional Medicine(湖北省精准放射与介入医学临床医学研究中心);The First Affiliated Hospital of Zhengzhou University(郑州大学第一附属医院);Huazhong University of Science and Technology(华中科技大学);The First Affiliated Hospital of Henan University of Science and Technology(河南科技大学第一附属医院)
Zero-Shot End-to-End Relation Extraction in Chinese: A Comparative Study of Gemini, LLaMA and ChatGPT
University of Amsterdam(阿姆斯特丹大学);Johns Hopkins University(约翰斯·霍普金斯大学);University of California San Diego(加利福尼亚大学圣迭戈分校);Simon Fraser University(西蒙菲莎大学);Northeastern University(东北大学)
Large Language Models for Education and Research: An Empirical and User Survey-based Analysis
Tulane University, New Orleans, LA, USA Dhaka University of Engineering \& Technology, Gazipur, Bangladesh The University of Aizu, Aizu-Wakamatsu, Fukushima, Japan
General-purpose AI models can generate actionable knowledge on agroecological crop protection
Chrysalis Consulting(Chrysalis咨询公司);Institute for Plant Protection, China Academy of Agricultural Sciences(中国农业科学院植物保护研究所);School of the Environment, University of Queensland(昆士兰大学环境学院)
A comprehensive study of LLM-based argument classification: from LLAMA through GPT-4o to Deepseek-R1
Faculty of Computer Science, Electronics, and Telecommunications AGH University of Krakow(计算机科学、电子与电信学院AGH克拉科夫大学);Faculty of Humanities AGH University of Krakow(人文学院AGH克拉科夫大学);Institute of Mathematics University of Silesia(数学学院桑特学院)
Performance Evaluation of Lightweight Open-source Large Language Models in Pediatric Consultations: A Comparative Analysis
Children’s Hospital of Chongqing Medical University(重庆医科大学附属儿童医院);Stanford University School of Medicine(斯坦福大学医学院);Chongqing University of Technology(重庆理工大学);Columbia University(哥伦比亚大学)
Convergences and Divergences between Automatic Assessment and Human Evaluation: Insights from Comparing ChatGPT-Generated Translation and Neural Machine Translation
School of Foreign Languages, Shanghai Jiao Tong University(上海交通大学外国语学院);Department of Computer Science and Engineering, Shanghai Jiao Tong University(上海交通大学计算机科学与工程系);Institute of Corpus Studies and Applications, Shanghai International Studies University(上海外国语大学语料库研究与应用研究所)
Comparative analysis of privacy-preserving open-source LLMs regarding extraction of diagnostic information from clinical CMR imaging reports
The D-Lab, Department of Precision Medicine, GROW - Research Institute for Oncology and Reproduction, Maastricht University(D实验室,精准医学系,GROW癌症与生殖研究研究所,马斯特里赫特大学);Division of Cardiovascular Imaging, Department of Cardiology I, University Hospital Münster(心血管影像科,心内科第一部门,穆恩斯特大学医院)
Intelligent Materials Modelling: Large Language Models Versus Partial Least Squares Regression for Predicting Polysulfone Membrane Mechanical Performance