在机器学习课程中利用Wiki LLM索引增强学习潜力的研究

中文摘要 AI 辅助

大型语言模型越来越多地被部署为课程专属辅导工具，但其有效性取决于对经过审核的教学材料的锚定，而这些材料通常会在学期中修订。我们先前的工作构建了一个基于多模态检索增强生成（RAG）系统，应用于真实的机器学习课程语料库（《机器学习基础》），并发现检索改善了上下文锚定，但固定的检索策略并非最优。这引出了另一个问题：语料库在摄入时的结构化方式是否比查询时的检索量更为重要。我们对同一课堂语料库上的两种知识表示进行了受控的面对面比较：（A）向量RAG，复现了我们先前研究中表现最佳的配置；（B）LLM编译的wiki（Karpathy框架），其中语料库在摄入时被综合成带有显式交叉引用和指向源材料引用的链接概念页面。我们评估了59个问题，涵盖单事实回忆、跨单元概念链接、综合与解释，以及教学大纲修订后的时效性，由LLM评判员根据人工编写的评分标准进行评分。两种表示在回答单事实问题上的表现几乎相同（9.33对9.96，满分10分），但在需要跨课程单元链接的问题上差异显著。编译后的wiki保持了准确性和锚定性（9.93；100%锚定于引用来源），而检索得分较低且锚定性明显不足（8.14；64%）。wiki的引用使学生和教师能够将任何主张追溯到引入该主张的讲座，增加了机器学习课程所需的动态检索层。虽然还需要进一步测试，但使用AI支持机器学习课程学习的教师应考虑基于wiki的结构，因为它有潜力支持最佳实践的基础要素。

英文摘要

Large language models are increasingly deployed as course-specific tutors, but their usefulness depends on grounding in vetted instructional materials that are often revised mid-semester. Our prior work built a multimodal retrieval-augmented generation (RAG) system over an authentic machine learning course corpus (Foundations of Machine Learning) and found that retrieval improved contextual grounding, but that fixed retrieval strategies were suboptimal. That motivates a different question: whether how a corpus is structured at ingest time matters more than how much is retrieved at query time. We present a controlled head-to-head comparison of two knowledge representations over an identical classroom corpus: (A) vector RAG, replicating the best-performing configuration from our prior study, and (B) an LLM-compiled wiki (Karpathy framework), in which the corpus is synthesized at ingest into linked concept pages with explicit cross-references and citations back to source materials. We evaluate 59 questions spanning single-fact recall, cross-unit concept linking, synthesis and explanation, and currency after a syllabus revision, scored by an LLM judge against a human-authored rubric. Both representations answered single-fact questions about equally well (9.33 vs. 9.96 of 10), but diverged sharply on questions requiring links across course units. The compiled wiki remained accurate and grounded (9.93; 100% grounded in cited sources), while retrieval scored lower and was markedly less grounded (8.14; 64%). The wiki's citations let students and instructors trace any claim back to the lecture that introduced it, adding a layer of dynamic retrieval that machine learning courses require. While further testing is needed, instructors using AI to support learning in ML courses should consider wiki-based structure for its potential to support foundational elements of best practice.

发表机构

• School of Data Science, University of Virginia（弗吉尼亚大学数据科学学院）

机构由 AI 辅助整理，请以论文原文为准。