Developing questions that are pedagogically sound, relevant, and promote learning is a challenging and time-consuming task for educators. Modern-day large language models (LLMs) generate high-quality content across multiple domains, potentially helping educators to develop high-quality questions. Automated educational question generation (AEQG) is important in scaling online education catering to a diverse student population. Past attempts at AEQG have shown limited abilities to generate questions at higher cognitive levels. In this study, we examine the ability of five state-of-the-art LLMs of different sizes to generate diverse and high-quality questions of different cognitive levels, as defined by Bloom's taxonomy. We use advanced prompting techniques with varying complexity for AEQG. We conducted expert and LLM-based evaluations to assess the linguistic and pedagogical relevance and quality of the questions. Our findings suggest that LLms can generate relevant and high-quality educational questions of different cognitive levels when prompted with adequate information, although there is a significant variance in the performance of the five LLms considered. We also show that automated evaluation is not on par with human evaluation.
OpenLearnLM Benchmark: A Unified Framework for Evaluating Knowledge, Skill, and Attitude in Educational Large Language Models
Chosun University(chosun大学);Korea University(韩国大学);Ewha Womans University(成均馆大学);Korea Institute for Curriculum and Evaluation(韩国课程评价院);Seoul National University(首尔国立大学);Texas A&M University(德克萨斯农工大学);Indiana University Bloomington(印第安纳大学布卢明顿分校)
BloomWise: Enhancing Problem-Solving capabilities of Large Language Models using Bloom's-Taxonomy-Inspired Prompts
Speech and Language Processing Group, National Technical University of Athens(国家技术大学雅典语音与语言处理组);Institute for Language and Speech Processing, Athena Research Center(雅典研究中心语言与语音处理研究所);National Technical University of Athens(国家技术大学雅典)
FairytaleQA Translated: Enabling Educational Question and Answer Generation in Less-Resourced Languages
Artificial Intelligence and Computer Science Laboratory (LIACC) Faculty of Engineering of the University of Porto (FEUP)(人工智能与计算机科学实验室(LIACC)工程学院(FEUP))
From Memorization to Creation: Evaluating the Cognitive Depth of LLM-Generated Educational Questions
City University of Hong Kong(香港城市大学);Zhejiang Normal University(浙江师范大学);Squirrel Ai Learning;University of Science and Technology of China(中国科学技术大学);Wuhan University(武汉大学)
Toward LLM-Supported Automated Assessment of Critical Thinking Subskills
University of Minnesota(明尼苏达大学);University of Massachusetts Amherst(马萨诸塞大学阿默斯特分校);Adelaide University(阿德莱德大学);Brighter Research(Brighter研究公司);Educational Testing Service (ETS)(教育考试服务中心(ETS))
Benchmarking Large Language Models for Diagnosing Students' Cognitive Skills from Handwritten Math Work
School of Computing, KAIST, Daejeon, Republic of Korea(韩国庆熙大学计算机学院);University of Michigan, Ann Arbor, USA(美国密歇根大学);Ewha Womans University, Seoul, Republic of Korea(韩国成均馆大学);AlgorithmLabs, Seoul, Republic of Korea(韩国AlgorithmLabs公司)
Are Video Models Zero-Shot Learners and Reasoners in Education? EduVideoBench, A Knowledge-Skills-Attitude Benchmark for Educational Video Generation
Korea University Sejong Campus(韩国大学世宗校区);Cardiff Metropolitan University(卡迪夫 Metropolitan 大学);Seoul National University(首尔国立大学);Bugil Academy(Bugil 学院);Gyeonggi Provincial Office of Education(京畿省教育厅);Loughborough University(洛桑大学);Korea National University of Education(韩国教育大学);Korea University(韩国大学);Korean Educational Development Institute(韩国教育发展院);Sungshin Women’s University(顺天堂女子大学);Seoul National University of Education(首尔国立教育大学);Korea Institute for Curriculum and Evaluation(韩国课程评价院);Indiana University Bloomington(印第安纳大学布卢明顿分校)
Enhancing AI-Driven Education: Integrating Cognitive Frameworks, Linguistic Feedback Analysis, and Ethical Considerations for Improved Content Generation
Learning, Data and Robotics (LDR) ESIEA Lab(学习、数据与机器人(LDR)ESIEA实验室);Department of Computer Engineering(计算机工程系);Graduate School of Management and Innovation(管理与创新研究生院);Faculty of Science(科学学院);Lebanese University(黎巴嫩大学)