并非所有选项都生而平等:面向令牌高效的大语言模型知识追踪的文本选项加权
Not All Options Are Created Equal: Textual Option Weighting for Token-Efficient LLM-Based Knowledge Tracing
AI总结:
本文提出LOKT框架,通过将示例学习者交互历史编码为文本类别选项权重(TCOW),在令牌约束下实现可扩展且高效的大语言模型知识追踪,并在冷启动和热启动设置中均优于现有模型。
摘要:
大语言模型(LLMs)因其强大的推理和泛化能力,近年来已成为知识追踪(KT)中颇具前景的工具。尽管近期基于LLM的知识追踪方法提出了新的提示格式,但它们在上下文学习(ICL)过程中难以在单个提示内表示示例学习者的完整交互历史,导致在令牌约束下可扩展性有限且计算成本高昂。在本工作中,我们提出了基于LLM的选项加权知识追踪(LOKT),这是一个简单而有效的框架,将示例学习者在上下文中的交互历史编码为文本类别选项权重(TCOW)。TCOW是分配给学习者在回答问题时所选选项的语义标签(例如“不足”),从而增强了LLM的可解释性。在多项选择题数据集上的实验表明,LOKT在冷启动和热启动设置下均优于现有的非LLM和基于LLM的知识追踪模型。此外,LOKT实现了可扩展且成本高效的推理,即使在严格的令牌约束下也能取得强劲性能。我们的代码可在https://anonymous.4open.science/r/LOKT_model-3233获取。
英文摘要:
Large Language Models (LLMs) have recently emerged as promising tools for knowledge tracing (KT) due to their strong reasoning and generalization abilities. While recent LLM-based KT methods have proposed new prompt formats, they struggle to represent the full interaction histories of example learners within a single prompt during in-context learning (ICL), resulting in limited scalability and high computational cost under token constraints. In this work, we present \textit{LLM-based Option-weighted Knowledge Tracing (LOKT)}, a simple yet effective framework that encodes the interaction histories of example learners in context as \textit{textual categorical option weights (TCOW)}. TCOW are semantic labels (e.g., ``inadequate'') assigned to the options selected by learners when answering questions, enhancing the interpretability of LLMs. Experiments on multiple-choice datasets show that LOKT outperforms existing non-LLM and LLM-based KT models in both cold-start and warm-start settings. Moreover, LOKT enables scalable and cost-efficient inference, achieving strong performance even under strict token constraints. Our code is available at \href{https://anonymous.4open.science/r/LOKT_model-3233}{https://anonymous.4open.science/r/LOKT\_model-3233}.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。