arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2310.20105cs.CYcs.AIcs.CL
收藏

使用大型语言模型对编程课程中学生求助请求进行高效分类

Efficient Classification of Student Help Requests in Programming Courses Using Large Language Models

Carnegie Mellon University(卡内基梅隆大学);The University of Auckland(奥克兰大学);Illinois Wesleyan University(伊利诺伊卫斯理大学)

Jaromir Savelka, Paul Denny, Mark Liffiton, Brad Sheese

AI总结:

本文研究利用GPT-3.5和GPT-4自动分类编程课程学生求助请求,发现GPT-4在调试子类上更优,微调GPT-3.5可接近人工评分一致性,证明LLM可用于教育需求自动分类。

摘要:

根据学生所寻求的帮助类型对其求助请求进行准确分类,有助于定制有效的回应。自动分类此类请求并非易事,但大型语言模型(LLMs)似乎提供了一种易于获取且具有成本效益的解决方案。本研究评估了GPT-3.5和GPT-4模型在一门编程入门课程中对学生求助请求进行分类的性能。在零样本试验中,GPT-3.5和GPT-4在大多数类别上表现出相当的性能,而GPT-4在分类与调试相关的子类别请求时优于GPT-3.5。对GPT-3.5模型进行微调后,其性能提升到接近两位人工评分者之间所观察到的跨类别准确性和一致性的程度。总体而言,本研究证明了利用大型语言模型通过自动分类学生需求来增强教育系统的可行性。

英文摘要:

The accurate classification of student help requests with respect to the type of help being sought can enable the tailoring of effective responses. Automatically classifying such requests is non-trivial, but large language models (LLMs) appear to offer an accessible, cost-effective solution. This study evaluates the performance of the GPT-3.5 and GPT-4 models for classifying help requests from students in an introductory programming class. In zero-shot trials, GPT-3.5 and GPT-4 exhibited comparable performance on most categories, while GPT-4 outperformed GPT-3.5 in classifying sub-categories for requests related to debugging. Fine-tuning the GPT-3.5 model improved its performance to such an extent that it approximated the accuracy and consistency across categories observed between two human raters. Overall, this study demonstrates the feasibility of using LLMs to enhance educational systems through the automated classification of student needs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑