Knowledge tracing (KT) models aim to predict students' future performance based on their historical interactions. Most existing KT models rely exclusively on human-defined knowledge concepts (KCs) associated with exercises. As a result, the effectiveness of these models is highly dependent on the quality and completeness of the predefined KCs. Human errors in labeling and the cost of covering all potential underlying KCs can limit model performance.
In this paper, we propose a KT model, Sparse Binary Representation KT (SBRKT), that generates new KC labels, referred to as auxiliary KCs, which can augment the predefined KCs to address the limitations of relying solely on human-defined KCs. These are learned through a binary vector representation, where each bit indicates the presence (one) or absence (zero) of an auxiliary KC. The resulting discrete representation allows these auxiliary KCs to be utilized in training any KT model that incorporates KCs. Unlike pre-trained dense embeddings, which are limited to models designed to accept such vectors, our discrete representations are compatible with both classical models, such as Bayesian Knowledge Tracing (BKT), and modern deep learning approaches.
To generate this discrete representation, SBRKT employs a binarization method that learns a sparse representation, fully trainable via stochastic gradient descent. Additionally, SBRKT incorporates a recurrent neural network (RNN) to capture temporal dynamics and predict future student responses by effectively combining the auxiliary and predefined KCs. Experimental results demonstrate that SBRKT outperforms the tested baselines on several datasets and achieves competitive performance on others. Furthermore, incorporating the learned auxiliary KCs consistently enhances the performance of BKT across all tested datasets.
How a Bilingual LM Becomes Bilingual: Tracing Internal Representations with Sparse Autoencoders
MBZUAI（穆萨大学人工智能研究所）；SOKENDAI（神户大学）；NINJAL（日本国家研究所）；Tohoku University（东北大学）；RIKEN（日本研究机构）；NII LLMC（日本信息处理学会大语言模型中心）；University of Tokyo（东京大学）；Nagoya Institute of Technology（名古屋技术大学）
Towards Robust Knowledge Tracing Models via k-Sparse Attention
Think Academy International Education（学而思国际教育）；TAL Education Group（好未来教育集团）；Guangdong Institute of Smart Education（广东省智慧教育研究院）；Jinan University（暨南大学）；City University of Hong Kong（香港城市大学）；College of Cyber Security（网络空间安全学院）
Tracing the Thought of a Grandmaster-level Chess-Playing Transformer
Shanghai Innovation Institute, Shanghai, China（上海创新研究院）；School of Computer Science, Fudan University, Shanghai, China（复旦大学计算机科学技术学院）；School of Mathematics and Statistics, Xi’an Jiaotong University（西安交通大学数学与统计学院）
Expert-Aware Causal Tracing of Factual Recall in Sparse MoE Language Models
Center for Information and Language Processing (CIS)（信息与语言处理中心）；Ubiquitous Knowledge Processing Lab (UKP)（无所不在的知识处理实验室）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）
HadamRNN: Binary and Sparse Ternary Orthogonal RNNs
Institut de Mathématiques de Toulouse（图卢兹数学研究所）；Institut de Recherche Technologique Saint Exupéry（圣埃克苏佩里技术研究所）；Université de Toulouse（图卢兹大学）；CNRS（法国国家科学研究中心）
Predicting Infectiousness for Proactive Contact Tracing
Mila（米拉研究所）；University of Oxford（牛津大学）；Max-Planck Institute for Intelligent Systems Tübingen（马克斯·普朗克智能系统研究所蒂宾根分所）；University of Ottawa（渥太华大学）；McGill University（麦吉尔大学）；Université de Montreal（蒙特利尔大学）；École Polytechnique de Montreal（蒙特利尔理工学院）；The Alan Turing Institute（阿兰·图灵研究所）；HEC Montréal（蒙特利尔高等商学院）
Data Augmentation for Sparse Multidimensional Learning Performance Data Using Generative AI
Institute for Intelligent Systems, University of Memphis（孟菲斯大学智能系统研究所）；Arkansas State University（阿肯色州立大学）；Human-Computer Interaction Institute, Carnegie Mellon University（卡内基梅隆大学人机交互研究所）；Faculty of Education, The University of Hong Kong（香港大学教育学院）；Centre for Learning Analytics, Faculty of Information Technology, Monash University（莫纳什大学信息技术学院学习分析中心）；Department of Applied Social Sciences, Hong Kong Polytechnic University（香港理工大学应用社会科学系）
