为电影工程专业设计的AI助教
An AI Teaching Assistant for Motion Picture Engineering
AI总结：
本文介绍基于RAG技术的AI助教在电影工程硕士课程中的应用，通过实验证明其在教学环境中的有效性，并展示学生反馈和评估结果。
摘要：
近年来，大语言模型的迅速发展推动了基于LLM的AI导师的实验。然而，实现细节和教学环境中的效益仍处于早期探索阶段。本文在特里尼蒂大学杜伦学院电影工程硕士课程的背景下，探讨了AI助教（AI-TA）的实现问题。我们提供了实现细节（包括提示词和代码），并强调了如何设计和调优RAG管道以满足课程需求。我们描述了我们的调查工具，并通过多个定量指标报告了AI-TA的影响。实验规模（43名学生，296次会话，1,889次查询，持续7周）足以使我们的发现具有说服力。与以往研究不同，我们试验了允许在开卷考试中使用AI-TA。统计分析显示，无论是否使用AI-TA，成绩均无差异（p > 0.05），证明精心设计的评估可以保持学术有效性。学生反馈显示，AI-TA是有益的（均值=4.22/5），而学生对AI-TA优于人类辅导的看法则较为混合（均值=2.78/5）。
英文摘要：
The rapid rise of LLMs over the last few years has promoted growing experimentation with LLM-driven AI tutors. However, the details of implementation, as well as the benefit in a teaching environment, are still in the early days of exploration. This article addresses these issues in the context of implementation of an AI Teaching Assistant (AI-TA) using Retrieval Augmented Generation (RAG) for Trinity College Dublin's Master's Motion Picture Engineering (MPE) course. We provide details of our implementation (including the prompt to the LLM, and code), and highlight how we designed and tuned our RAG pipeline to meet course needs. We describe our survey instrument and report on the impact of the AI-TA through a number of quantitative metrics. The scale of our experiment (43 students, 296 sessions, 1,889 queries over 7 weeks) was sufficient to have confidence in our findings. Unlike previous studies, we experimented with allowing the use of the AI-TA in open-book examinations. Statistical analysis across three exams showed no performance differences regardless of AI-TA access (p > 0.05), demonstrating that thoughtfully designed assessments can maintain academic validity. Student feedback revealed that the AI-TA was beneficial (mean = 4.22/5), while students had mixed feelings about preferring it over human tutoring (mean = 2.78/5).
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
