The increasing demand for programming language education and growing class sizes require immediate and personalized feedback. However, traditional code review methods have limitations in providing this level of feedback. As the capabilities of Large Language Models (LLMs) like GPT for generating accurate solutions and timely code reviews are verified, this research proposes a system that employs GPT-4 to offer learner-friendly code reviews and minimize the risk of AI-assist cheating.
To provide learner-friendly code reviews, a dataset was collected from an online judge system, and this dataset was utilized to develop and enhance the system's prompts. In addition, to minimize AI-assist cheating, the system flow was designed to provide code reviews only for code submitted by a learner, and a feature that highlights code lines to fix was added. After the initial system was deployed on the web, software education experts conducted usability test. Based on the results, improvement strategies were developed to improve code review and code correctness check module, thereby enhancing the system.
The improved system underwent evaluation by software education experts based on four criteria: strict code correctness checks, response time, lower API call costs, and the quality of code reviews. The results demonstrated a performance to accurately identify error types, shorten response times, lower API call costs, and maintain high-quality code reviews without major issues. Feedback from participants affirmed the tool's suitability for teaching programming to primary and secondary school students. Given these benefits, the system is anticipated to be a efficient learning tool in programming language learning for educational settings.
Large Language Models (GPT) for automating feedback on programming assignments
Institute of Information Technology, Warsaw University of Life Sciences（华沙生命科学大学信息技术研究所）；Penn Center for Learning Analytics, University of Pennsylvania（宾夕法尼亚大学宾夕法尼亚学习分析中心）
Compiler-Guided Inference-Time Adaptation: Improving GPT-5 Programming Performance in Idris
Ming Hsieh Department of Electrical and Computer Engineering（明希斯部门电子与计算机工程系）；Viterbi School of Engineering, University of Southern California（维特比工程学院，南加州大学）
A-ProS: Towards Reliable Autonomous Programming Through Multi-Model Feedback
Dept. of Computer Science and Engineering, University of Dhaka（达卡大学计算机科学与工程系）；Dept. of Information Systems, University of Maryland, Baltimore County（马里兰大学巴尔的摩县信息学院）；University of Maryland, Baltimore County（马里兰大学巴尔的摩县）
Can large language models replace humans in the systematic review process? Evaluating GPT-4's efficacy in screening and extracting data from peer-reviewed and grey literature in multiple languages
Trinity Centre for Global Health（三一全球健康中心）；School of Psychology（心理学院）；Trinity College Dublin（都柏林圣三一学院）；University of York（约克大学）
Catch Me If You Can: Identifying Fraudulent Physician Reviews with Large Language Models Using Generative Pre-Trained Transformers
Beedie School of Business, Simon Fraser University（西蒙菲莎大学比迪商学院）；Faculty of Applied Science, Simon Fraser University（西蒙菲莎大学应用科学学院）；Carey Business School, John Hopkins University（约翰霍普金斯大学凯瑞商学院）
Strategic Optimization and Challenges of Large Language Models in Object-Oriented Programming
Institute for Clarity in Documentation（文献清晰度研究所）；Inria Paris-Rocquencourt（法国国家信息与自动化研究所巴黎-罗康库尔中心）；Rajiv Gandhi University（拉吉夫·甘地大学）；Tsinghua University（清华大学）；Palmer Research Laboratories（帕尔默研究实验室）
Fooling MOSS Detection with Pretrained Language Models
Booz Allen Hamilton（博思艾伦咨询公司）；EleutherAI（EleutherAI 人工智能研究组织）
随着人工智能（AI）技术在社会中变得越来越强大和突出，它们的滥用越来越令人担忧。在教育环境中，人工智能技术可以被学生用来在作业和考试中作弊。在本文中，我们将探讨Transformers是否可以用于解决入门级编程任务，同时绕过常用的AI工具来检测软件之间的相似性。我们发现，使用GPT-J的学生[Wang and Komatsuzaki，2021]可以完成入门级的编程作业，而不会引起MOSS [Aiken，2000]的怀疑，MOSS是一种广泛使用的软件相似性和剽窃检测工具。尽管GPT-J没有接受过有关问题的培训，也没有提供任何可供借鉴的例子，但情况仍然如此。我们进一步发现，GPT-J编写的代码在结构上是多样的，缺乏任何特定的告诉，未来的剽窃检测技术可能会用来尝试识别算法生成的代码。最后，我们讨论的伦理和教育的影响，大语言模型和未来的研究方向。
Towards Safer Chatbots: Automated Policy Compliance Evaluation of Custom GPTs
Information Processing and Telecommunications Center Universidad Politécnica de Madrid ETSI Telecomunicación（信息处理与电信中心西班牙马德里理工大学电信工程学院）；King’s College London（伦敦国王学院）；INGENIO (CSIC-Universitat Politècnica de València)（INGENIO（西班牙国家研究委员会-瓦伦西亚理工大学））
