This paper examines the potential for generative artificial intelligence (GenAI) to assist with internal review processes for research quality evaluations in UK higher education and particularly in preparation for the Research Excellence Framework (REF). Using the lens of function substitution in the Viable Systems Model, we present an experimental methodology using ChatGPT to score and rank business and management papers from REF 2021 submissions, "reverse engineering" the assessment by comparing AI-generated scores with known institutional results. Through rigourous testing of 822 papers across 11 institutions, we established scoring boundaries that aligned with reported REF outcomes: 49% between 1* and 2*, 59% between 2* and 3*, and 69% between 3* and 4*. The results demonstrate that AI can provide consistent evaluations that help identify borderline evaluation cases requiring additional human scrutiny while reducing the substantial resource burden of traditional internal review processes. We argue for application through a nuanced hybrid approach that maintains academic integrity while addressing the multi-million pound costs associated with research evaluation bureaucracy. While acknowledging these limitations including potential AI biases, the research presents a promising framework for more efficient, consistent evaluations that could transform current approaches to research assessment.
LLM-assisted sentiment analysis for integrated computational and qualitative mixed methods education research: A case study of students' written reflection assignments
Chandra Family Department of Electrical and Computer Engineering, University of Texas at Austin(德克萨斯大学奥斯汀分校电子与计算机工程系);Department of Engineering Education, Virginia Polytechnic Institute and State University(弗吉尼亚理工大学工程教育系);Gallogly College of Engineering, University of Oklahoma(俄克拉荷马大学加洛格利工程学院);Department of Mechanical and Materials Engineering, University of Nebraska-Lincoln(内布拉斯加大学林肯分校机械与材料工程系)
Evaluating AI-Powered Learning Assistants in Engineering Higher Education: Student Engagement, Ethical Challenges, and Policy Implications
ByWater Institute, Tulane University(杜兰大学拜沃特研究所);Office of Teaching, Learning, and Technology, University of Iowa(爱荷华大学教学、学习与技术办公室);River-Coastal Science and Engineering, Tulane University(杜兰大学河流与海岸科学与工程系)
本研究评估了在R1公立大学工程本科课程中应用的AI学习框架Educational AI Hub,通过混合方法发现学生认可其便利性但存在伦理顾虑,强调需提升可用性、伦理透明度并辅以教师指导以促进有效AI学习参与。
Who Acts, Who Knows, Who Answers? A Corpus-Assisted Discourse Analysis of Agency, Epistemic Responsibility, and Accountability in Generative AI Higher Education Research
Can LLMs Assist Computer Education? an Empirical Case Study of DeepSeek
Faculty of Data Science, City University of Macau(澳门大学数据科学学院);Design and Creative Technology Vertical, Torrens University Australia(澳大利亚托伦斯大学设计与创意技术垂直领域)
Integrating Generative AI into Cybersecurity Education: A Study of OCR and Multimodal LLM-assisted Instruction
College of Information Science, University of Arizona(亚利桑那大学信息科学学院);Department of Electrical and Computer Engineering, University of Arizona(亚利桑那大学电气与计算机工程系);Department of Systems and Industrial Engineering, University of Arizona(亚利桑那大学系统与工业工程系);Department of Industrial Engineering, University of Sonora(索诺拉大学工业工程系);Leonhard Center for Enhancement of Engineering Education, The Pennsylvania State University(宾夕法尼亚州立大学莱昂哈德工程教育提升中心)