arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2408.04394cs.CLcs.AI
收藏

使用大型语言模型在不同布鲁姆技能水平下自动生成教育问题:策略与评估

Automated Educational Question Generation at Different Bloom's Skill Levels using Large Language Models: Strategies and Evaluation

Indian Institute of Science(印度科学学院);VIT-AP University(VIT-AP大学)

Nicy Scaria, Suma Dharani Chenna, Deepak Subramani

AI总结:

本研究考察了五种不同规模的最先进大型语言模型结合高级提示技术,生成不同布鲁姆认知水平教育问题的能力,发现LLMs在信息充足时能生成高质量问题,但模型间性能差异显著且自动评估不及人工评估。

摘要:

为教育者开发符合教学法、相关且能促进学习的问题是一项具有挑战性且耗时的任务。现代大型语言模型(LLMs)能够跨多个领域生成高质量内容,有望帮助教育者开发高质量问题。自动教育问题生成(AEQG)在扩展面向多样化学生群体的在线教育方面具有重要意义。以往对AEQG的尝试在生成较高认知水平的问题方面能力有限。在本研究中,我们考察了五种不同规模的最先进LLMs生成由布鲁姆分类法定义的不同认知水平的多样化、高质量问题的能力。我们使用了复杂度不同的高级提示技术进行AEQG。我们进行了专家评估和基于LLM的评估,以评估问题的语言学和教学法相关性及质量。研究结果表明,当提供充足信息的提示时,LLMs能够生成不同认知水平的相关且高质量的教育问题,尽管所考虑的五种LLMs在性能上存在显著差异。我们还表明,自动评估尚未达到与人工评估同等的水平。

英文摘要:

Developing questions that are pedagogically sound, relevant, and promote learning is a challenging and time-consuming task for educators. Modern-day large language models (LLMs) generate high-quality content across multiple domains, potentially helping educators to develop high-quality questions. Automated educational question generation (AEQG) is important in scaling online education catering to a diverse student population. Past attempts at AEQG have shown limited abilities to generate questions at higher cognitive levels. In this study, we examine the ability of five state-of-the-art LLMs of different sizes to generate diverse and high-quality questions of different cognitive levels, as defined by Bloom's taxonomy. We use advanced prompting techniques with varying complexity for AEQG. We conducted expert and LLM-based evaluations to assess the linguistic and pedagogical relevance and quality of the questions. Our findings suggest that LLms can generate relevant and high-quality educational questions of different cognitive levels when prompted with adequate information, although there is a significant variance in the performance of the five LLms considered. We also show that automated evaluation is not on par with human evaluation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑