自 2022 年 11 月发布以来,ChatGPT 在用户中越来越受欢迎,尤其是程序员将其作为解决开发问题的工具。然而,尽管 ChatGPT 为编程问题提供了实用的解决方案,但它主要应作为辅助工具(例如在软件教育中)使用,而不是替代人类。因此,检测由 ChatGPT 自动生成的源代码是必要的,用于识别 AI 生成内容的工具可能需要调整以有效处理源代码。本文提出了一项实证研究,以探究自动识别 AI 生成代码片段的可行性,以及影响该能力的因素。为此,我们提出了一种名为 GPTSniffer 的新方法,该方法基于 CodeBERT 构建,用于检测由 AI 编写的源代码。结果表明,GPTSniffer 能够准确分类代码是人类编写还是 AI 生成,并且优于两个基线方法 GPTZero 和 OpenAI Text Classifier。此外,研究还表明,相似的训练数据或带有成对代码片段的分类上下文有助于提升分类性能。
英文摘要:
Since its launch in November 2022, ChatGPT has gained popularity among users, especially programmers who use it as a tool to solve development problems. However, while offering a practical solution to programming problems, ChatGPT should be mainly used as a supporting tool (e.g., in software education) rather than as a replacement for the human being. Thus, detecting automatically generated source code by ChatGPT is necessary, and tools for identifying AI-generated content may need to be adapted to work effectively with source code. This paper presents an empirical study to investigate the feasibility of automated identification of AI-generated code snippets, and the factors that influence this ability. To this end, we propose a novel approach called GPTSniffer, which builds on top of CodeBERT to detect source code written by AI. The results show that GPTSniffer can accurately classify whether code is human-written or AI-generated, and outperforms two baselines, GPTZero and OpenAI Text Classifier. Also, the study shows how similar training data or a classification context with paired snippets helps to boost classification performances.
Improving Stack Overflow question title generation with copying enhanced CodeBERT model and bi-modal information
Wuhan University of Technology(武汉理工大学);Wuhan University(武汉大学);Wuhan University of Technology Chongqing Research Institute(武汉理工大学重庆研究院);Sanya Science and Education Innovation Park of Wuhan University of Technology(武汉理工大学三亚科教创新园);City University of Hong Kong(香港城市大学)
CoCoSoDa: Effective Contrastive Learning for Code Search
Xi’an Jiaotong University(西安交通大学);School of Software Engineering, Sun Yat-sen University(中山大学软件工程学院);The Chinese University of Hong Kong(香港中文大学);Microsoft Research(微软研究院);Chongqing University(重庆大学)
An Empirical Study of Foundation Models for Variability-Induced Compilation Errors in Configurable C Code
Federal University of Campina Grande (UFCG)(坎皮纳格兰德联邦大学);Federal University of Alagoas(阿拉戈阿斯联邦大学);Federal University of Bahia(巴伊亚联邦大学);VIRTUS Research, Development and Innovation Center/UFCG(VIRTUS研究与创新中心/坎皮纳格兰德联邦大学)