arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2108.04430cs.CYcs.LG
收藏

通过对抗性训练增强知识追踪

Enhancing Knowledge Tracing via Adversarial Training

Wangxuan Institute of Computer Technology, Peking University(北京大学王选计算机研究所)

Xiaopeng Guo, Zhijie Huang, Jie Gao, Mingyu Shang, Maojing Shu, Jun Sun

摘要:

我们研究的问题知识追踪(KT)的目标是跟踪学生的知识掌握随着时间的推移,以便对他们未来的表现作出预测。由于深度神经网络(DNN)具有良好的表示能力,KT的最新进展越来越多地集中在探索DNN以提高KT的性能。然而,我们的经验表明,基于DNN的KT模型可能存在过拟合的风险,特别是在小数据集上,导致泛化能力有限。在本文中,通过利用当前对抗训练(AT)的进展,我们提出了一种有效的基于AT的KT方法(ATKT)来增强KT模型的泛化能力,从而推动KT的极限。具体来说,我们首先构造对抗性扰动,并将它们作为对抗性示例添加到原始交互嵌入上。进一步利用原始样本和对抗样本对KT模型进行联合训练,使KT模型不仅对对抗样本具有鲁棒性,而且提高了模型的泛化能力。为了更好地实现AT,我们提出了一个高效的attentive-LSTM模型作为KT骨干,其中的关键是一个建议的知识隐藏状态注意力模块,该模块自适应地聚合来自先前知识隐藏状态的信息,同时突出当前知识隐藏状态的重要性,以进行更准确的预测。在四个公共基准数据集上的大量实验表明,我们的ATKT实现了新的最先进的性能。代码可从以下地址获得:\color{blue} { \url{https://github.com/xiaopengguo/ATKT} }。

英文摘要:

We study the problem of knowledge tracing (KT) where the goal is to trace the students' knowledge mastery over time so as to make predictions on their future performance. Owing to the good representation capacity of deep neural networks (DNNs), recent advances on KT have increasingly concentrated on exploring DNNs to improve the performance of KT. However, we empirically reveal that the DNNs based KT models may run the risk of overfitting, especially on small datasets, leading to limited generalization. In this paper, by leveraging the current advances in adversarial training (AT), we propose an efficient AT based KT method (ATKT) to enhance KT model's generalization and thus push the limit of KT. Specifically, we first construct adversarial perturbations and add them on the original interaction embeddings as adversarial examples. The original and adversarial examples are further used to jointly train the KT model, forcing it is not only to be robust to the adversarial examples, but also to enhance the generalization over the original ones. To better implement AT, we then present an efficient attentive-LSTM model as KT backbone, where the key is a proposed knowledge hidden state attention module that adaptively aggregates information from previous knowledge hidden states while simultaneously highlighting the importance of current knowledge hidden state to make a more accurate prediction. Extensive experiments on four public benchmark datasets demonstrate that our ATKT achieves new state-of-the-art performance. Code is available at: \color{blue} {\url{https://github.com/xiaopengguo/ATKT}}.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑