LSTM-SAKT：用于知识跟踪的LSTM编码的类SAKT Transformer
LSTM-SAKT: LSTM-Encoded SAKT-like Transformer for Knowledge Tracing
摘要：
本文介绍了Riiid！Kaggle是世界上最大的数据科学竞赛网站。本次比赛于2020年10月16日至2021年1月7日举行，共有3395支队伍、4387名选手参赛。本文的主要观点和贡献如下。(i)我们指出，现有的基于transformer的模型存在一个问题，即它们的查询/键/值可以包含的信息是有限的。为了解决这个问题，我们提出了一种使用LSTM获取查询/键/值的方法，并验证了其有效性。(ii)我们指出了“容器间”泄漏问题，这发生在问题有时一起提供的数据集中。为了解决这个问题，我们展示了特殊的索引/掩码技术，这些技术在使用RNN变体和Transformer时非常有用。(iii)我们发现额外的手工制作的功能可以有效地克服Transformer的限制，它永远不会考虑比序列长度更早的样本。
英文摘要：
This paper introduces the 2nd place solution for the Riiid! Answer Correctness Prediction in Kaggle, the world's largest data science competition website. This competition was held from October 16, 2020, to January 7, 2021, with 3395 teams and 4387 competitors. The main insights and contributions of this paper are as follows. (i) We pointed out existing Transformer-based models are suffering from a problem that the information which their query/key/value can contain is limited. To solve this problem, we proposed a method that uses LSTM to obtain query/key/value and verified its effectiveness. (ii) We pointed out 'inter-container' leakage problem, which happens in datasets where questions are sometimes served together. To solve this problem, we showed special indexing/masking techniques that are useful when using RNN-variants and Transformer. (iii) We found additional hand-crafted features are effective to overcome the limits of Transformer, which can never consider the samples older than the sequence length.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
