arXiv 2601.138822026-01-20cs.CL

OpenLearnLM基准：一个评估教育大型语言模型知识、技能和态度的统一框架

OpenLearnLM Benchmark: A Unified Framework for Evaluating Knowledge, Skill, and Attitude in Educational Large Language Models

Chosun University（chosun大学）；Korea University（韩国大学）；Ewha Womans University（成均馆大学）；Korea Institute for Curriculum and Evaluation（韩国课程评价院）；Seoul National University（首尔国立大学）；Texas A&M University（德克萨斯农工大学）；Indiana University Bloomington（印第安纳大学布卢明顿分校）

Unggi Lee, Sookbun Lee, Heungsoo Choi, Jinseo Lee, Haeun Park, Younghoon Jeon, Sungmin Cho, Minju Kang, Junbo Koh, Jiyeong Bae, Minwoo Nam, Juyeon Eun, Yeonji Jung, Yeil Jeong

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

OpenLearnLM基准通过统一框架评估教育大型语言模型的知识、技能和态度，揭示不同模型在多维度上的能力差异。

摘要：

大型语言模型正越来越多地被用作教育工具，但现有的基准测试侧重于狭窄的技能，缺乏对学习科学的支撑。我们介绍了OpenLearnLM基准，这是一个理论支撑的框架，用于评估LLM在三个维度上的表现，这些维度源自教育评估理论：知识（课程对齐的内容和教学法理解）、技能（基于场景的能力，通过四级中心-角色-场景-子场景层次结构组织）和态度（一致性对齐和欺骗抵抗）。我们的基准包括超过124,000个项目，涵盖多个学科、教育角色和难度级别，基于布卢姆分类法。知识领域优先考虑来自已建立基准的真实评估项目，而态度领域适应Anthropic的对齐欺骗方法，以检测在不同监控条件下行为的一致性不一致。对七个前沿模型的评估揭示了不同的能力轮廓：Claude-Opus-4.5在实际技能上表现突出，尽管内容知识较低，而Grok-4.1-fast在知识上领先，但存在对齐问题。值得注意的是，没有单一模型在所有维度上占优，验证了多轴评估的必要性。OpenLearnLM提供了一个开放、全面的框架，以促进LLM在真实教育情境中的准备。

英文摘要：

Large Language Models are increasingly deployed as educational tools, yet existing benchmarks focus on narrow skills and lack grounding in learning sciences. We introduce OpenLearnLM Benchmark, a theory-grounded framework evaluating LLMs across three dimensions derived from educational assessment theory: Knowledge (curriculum-aligned content and pedagogical understanding), Skills (scenario-based competencies organized through a four-level center-role-scenario-subscenario hierarchy), and Attitude (alignment consistency and deception resistance). Our benchmark comprises 124K+ items spanning multiple subjects, educational roles, and difficulty levels based on Bloom's taxonomy. The Knowledge domain prioritizes authentic assessment items from established benchmarks, while the Attitude domain adapts Anthropic's Alignment Faking methodology to detect behavioral inconsistency under varying monitoring conditions. Evaluation of seven frontier models reveals distinct capability profiles: Claude-Opus-4.5 excels in practical skills despite lower content knowledge, while Grok-4.1-fast leads in knowledge but shows alignment concerns. Notably, no single model dominates all dimensions, validating the necessity of multi-axis evaluation. OpenLearnLM provides an open, comprehensive framework for advancing LLM readiness in authentic educational contexts.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。