In this paper we promote introducing software verification and control flow graph similarity measurement in automated evaluation of students' programs. We present a new grading framework that merges results obtained by combination of these two approaches with results obtained by automated testing, leading to improved quality and precision of automated grading. These two approaches are also useful in providing a comprehensible feedback that can help students to improve the quality of their programs We also present our corresponding tools that are publicly available and open source. The tools are based on LLVM low-level intermediate code representation, so they could be applied to a number of programming languages. Experimental evaluation of the proposed grading framework is performed on a corpus of university students' programs written in programming language C. Results of the experiments show that automatically generated grades are highly correlated with manually determined grades suggesting that the presented tools can find real-world applications in studying and grading.
Large Language Models (GPT) for automating feedback on programming assignments
Institute of Information Technology, Warsaw University of Life Sciences(华沙生命科学大学信息技术研究所);Penn Center for Learning Analytics, University of Pennsylvania(宾夕法尼亚大学宾夕法尼亚学习分析中心)
LLM-assisted sentiment analysis for integrated computational and qualitative mixed methods education research: A case study of students' written reflection assignments
Chandra Family Department of Electrical and Computer Engineering, University of Texas at Austin(德克萨斯大学奥斯汀分校电子与计算机工程系);Department of Engineering Education, Virginia Polytechnic Institute and State University(弗吉尼亚理工大学工程教育系);Gallogly College of Engineering, University of Oklahoma(俄克拉荷马大学加洛格利工程学院);Department of Mechanical and Materials Engineering, University of Nebraska-Lincoln(内布拉斯加大学林肯分校机械与材料工程系)
Stepwise Verification and Remediation of Student Reasoning Errors with Large Language Model Tutors
ETH AI Center(ETH AI中心);TU Darmstadt(达姆施塔特工业大学);ETH Zurich(苏黎世联邦理工学院);Hessian Center for AI (hessian.AI)(黑森人工智能中心);Ubiquitous Knowledge Processing Lab (UKP Lab)(普适知识处理实验室);Department of Computer Science(计算机科学系);Professorship for Learning Sciences and Higher Education(学习科学与高等教育教授席位)
FARCA: Fact-Aligned Reliability-Aware Credit Assignment for Reinforcement Learning with Factual Supervision
School of Computer Science and Engineering, Nanjing University of Science and Technology(南京理工大学计算机科学与工程学院);School of Intelligence Science and Technology, Nanjing University(南京大学智能科学与技术学院)