基于提示的成本效益评估与运行：将ChatGPT用作计算机编程教学助手
Prompt-Based Cost-Effective Evaluation and Operation of ChatGPT as a Computer Programming Teaching Assistant
AI总结：
本文研究将ChatGPT用作编程教学助手的可行性，评估GPT-3.5T和GPT-4T的反馈表现，提出基于提示和上下文学习的自动化评估方法，并给出实用学习工具的实现策略。
摘要：
得益于大语言模型（LLMs）的出现，实现1:1师生比的梦想比以往任何时候都更加接近。这些模型在教育领域的一个潜在应用是向大学入门编程课程的学生提供反馈，使那些在解决基础实现问题时遇到困难的学生能够随时向全天候可用的LLM寻求帮助。本文重点研究与这一应用相关的三个方面。首先，评估了两个知名模型GPT-3.5T和GPT-4T在向学生提供反馈方面的表现。实证结果表明，GPT-4T的表现远优于GPT-3.5T，但它尚未准备好用于真实场景。这是因为其可能生成错误信息，而潜在用户并不总能察觉这些错误。其次，本文提出了一种精心设计的提示，利用上下文学习（in-context learning）技术，能够自动化评估过程的重要部分，并为包含错误信息的反馈比例提供一个下界，从而节省时间和精力。这之所以可行，是因为生成的反馈具有可编程分析的结构，其中包含了关于LLM在解决所请求任务时表现的诊断信息。第三，本文还提出了一种基于所提提示技术的、实现基于LLM的实用学习工具的可能策略。从教学角度看，这一策略开辟了一系列有趣的可能性。
英文摘要：
The dream of achieving a student-teacher ratio of 1:1 is closer than ever thanks to the emergence of large language models (LLMs). One potential application of these models in the educational field would be to provide feedback to students in university introductory programming courses, so that a student struggling to solve a basic implementation problem could seek help from an LLM available 24/7. This article focuses on studying three aspects related to such an application. First, the performance of two well-known models, GPT-3.5T and GPT-4T, in providing feedback to students is evaluated. The empirical results showed that GPT-4T performs much better than GPT-3.5T, however, it is not yet ready for use in a real-world scenario. This is due to the possibility of generating incorrect information that potential users may not always be able to detect. Second, the article proposes a carefully designed prompt using in-context learning techniques that allows automating important parts of the evaluation process, as well as providing a lower bound for the fraction of feedbacks containing incorrect information, saving time and effort. This was possible because the resulting feedback has a programmatically analyzable structure that incorporates diagnostic information about the LLM's performance in solving the requested task. Third, the article also suggests a possible strategy for implementing a practical learning tool based on LLMs, which is rooted on the proposed prompting techniques. This strategy opens up a whole range of interesting possibilities from a pedagogical perspective.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
