arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2403.11984cs.CLcs.AIcs.HC
收藏

使用生成式文本模型为教学评价创建定性编码本

Using Generative Text Models to Create Qualitative Codebooks for Student Evaluations of Teaching

Virginia Tech(弗吉尼亚理工大学)

Andrew Katz, Mitchell Gerhardt, Michelle Soledad

AI总结:

本文提出一种结合自然语言处理与大语言模型的新方法,用于分析大规模学生教学评价,通过提取、嵌入、聚类和总结来识别主题并生成编码本,并在5000份评价上验证其有效性。

摘要:

反馈是改进的关键方面。不幸的是,当来自多个来源的反馈很多时,将信息提炼为可操作的见解可能很困难。以学生教学评价(SETs)为例,它们是教育工作者重要的反馈来源。它们可以让教师了解在一个学期中哪些做法有效。一组SETs也可以作为课程或整个项目的信号,对管理人员有用。然而,在大规模情况下,如高注册人数课程或多年行政记录,SETs的数量可能使其难以分析。在本文中,我们讨论了一种使用自然语言处理(NLP)和大语言模型(LLMs)分析SETs的新方法。我们通过将该方法应用于一所大型公立大学的5,000份SETs语料库来演示该方法。我们表明,该方法可用于提取、嵌入、聚类和总结SETs,以识别它们表达的主题。更一般地,这项工作说明了如何使用NLP技术和LLMs的组合来为SETs生成编码本。最后,我们讨论了该方法在分析SETs以及教学和研究环境中其他类型学生写作方面的意义。

英文摘要:

Feedback is a critical aspect of improvement. Unfortunately, when there is a lot of feedback from multiple sources, it can be difficult to distill the information into actionable insights. Consider student evaluations of teaching (SETs), which are important sources of feedback for educators. They can give instructors insights into what worked during a semester. A collection of SETs can also be useful to administrators as signals for courses or entire programs. However, on a large scale as in high-enrollment courses or administrative records over several years, the volume of SETs can render them difficult to analyze. In this paper, we discuss a novel method for analyzing SETs using natural language processing (NLP) and large language models (LLMs). We demonstrate the method by applying it to a corpus of 5,000 SETs from a large public university. We show that the method can be used to extract, embed, cluster, and summarize the SETs to identify the themes they express. More generally, this work illustrates how to use the combination of NLP techniques and LLMs to generate a codebook for SETs. We conclude by discussing the implications of this method for analyzing SETs and other types of student writing in teaching and research settings.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑