arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2603.24073cs.CL
收藏

ConceptKT:一种用于知识追踪中概念层面缺陷预测的基准

ConceptKT: A Benchmark for Concept-Level Deficiency Prediction in Knowledge Tracing

Yu-Chen Kang, Yu-Chien Tang, An-Zi Yen

AI总结:

本文提出ConceptKT基准,通过标注问题所需概念和错误响应下的缺失概念,评估大语言模型和推理模型在概念层面缺陷预测中的性能。

摘要:

知识追踪(KT)是建模学生知识以支持个性化学习的关键技术。然而,大多数KT系统专注于二元正确性预测,无法诊断导致错误的底层概念误解。此类细粒度诊断反馈对于设计针对性教学和有效补救至关重要。本文引入了概念层面缺陷预测任务,通过识别学生未来问题中可能遇到困难的具体概念来扩展传统KT。我们提出了ConceptKT数据集,该数据集标注了每个问题所需的概念以及错误响应下的缺失概念。我们研究了上下文学习方法用于KT,并评估了各种大语言模型(LLMs)和大推理模型(LRMs)的诊断能力。探讨了不同选择信息历史记录的策略。实验结果表明,基于概念对齐和语义相似性选择响应历史记录能提高正确性预测和概念层面缺陷识别的性能。

英文摘要:

Knowledge Tracing (KT) is a critical technique for modeling student knowledge to support personalized learning. However, most KT systems focus on binary correctness prediction and cannot diagnose the underlying conceptual misunderstandings that lead to errors. Such fine-grained diagnostic feedback is essential for designing targeted instruction and effective remediation. In this work, we introduce the task of concept-level deficiency prediction, which extends traditional KT by identifying the specific concepts a student is likely to struggle with on future problems. We present ConceptKT, a dataset annotated with labels that capture both the concepts required to solve each question and the missing concepts underlying incorrect responses. We investigate in-context learning approaches to KT and evaluate the diagnostic capabilities of various Large Language Models (LLMs) and Large Reasoning Models (LRMs). Different strategies for selecting informative historical records are explored. Experimental results demonstrate that selecting response histories based on conceptual alignment and semantic similarity leads to improved performance on both correctness prediction and concept-level deficiency identification.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑