大型语言模型在高等教育课程评估中的探索
A Course Intelligence Platform for Higher Education: Lessons from AI-Assisted Course Evaluation
AI总结：
本研究利用大型语言模型进行高等教育课程评估，发现微调和提示工程能显著提高评估准确性，LLM生成的反馈有助于教学改进。
摘要：
课程评估在确保教学质量和指导课程发展方面起着至关重要的作用。然而，传统的评估方法，如学生调查、课堂观察和专家评审，往往受到主观性、高劳动力成本和有限可扩展性的限制。随着大型语言模型（LLMs）的最新进展，生成一致、细致和可扩展的课程评估的新机会出现了。本研究调查了使用三种代表性的LLMs进行自动化课程评估在微观层面（课堂讨论分析）和宏观层面（整体课程评审）的应用。利用课堂互动转录文本和一个包含100门课程的中国主要机构数据集，我们证明LLMs可以提取关键的教学法特征，并生成与专家判断一致的结构化评估结果。经过微调的Llama版本显示出更高的可靠性，其生成的分数分布具有更大的区分度，并且与人类评估者有更强的相关性。结果突显了三个主要发现：（1）LLMs可以在微观和宏观层面可靠地进行系统且可解释的课程评估；（2）微调和提示工程显著提高了评估的准确性和一致性；（3）LLM生成的反馈为教学改进提供了可行的见解。这些发现展示了基于LLM的评估作为支持大规模高等教育质量保证和教育决策的实用工具的潜力。
英文摘要：
The rapid adoption of generative AI has created new opportunities for teaching, learning, and quality assurance. Existing applications, however, remain largely student-facing, with comparatively limited attention to institution-level needs. This paper presents a course intelligence platform deployed across more than 100 universities and serving over 10,000 instructors in China. By linking competency requirements, knowledge structures, teaching activities, and assessment evidence, it establishes a shared foundation for knowledge organization, instructional design, learning assessment, and quality evaluation. The course evaluation module is examined as a representative institution-facing application of the platform, which integrates national evaluation standards, structured educational evidence, customized prompting strategies, and domain-adapted LLMs to generate quantitative scores and qualitative feedback. A case study involving 100 authentic university courses is conducted to evaluate its alignment with expert judgments and the interpretability of its outputs. Statistical analyses show substantial agreement between AI-generated assessments and expert ratings, while qualitative results highlight the credibility of the feedback. The findings further suggest that AI-assisted evaluation requires not only capable models but also structured domain knowledge and transparent criteria. In this context, human ratings should be treated as an informative reference rather than an error-free gold standard, and the objective is to achieve consistent, interpretable, and defensible judgments instead of merely replicating expert scores.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
