arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2310.14079cs.IRcs.AIcs.LG
收藏

复制,还是不复制;这是神经序列推荐器中输出 Softmax 层的一个关键问题

To Copy, or not to Copy; That is a Critical Issue of the Output Softmax Layer in Neural Sequential Recommenders

University of Massachusetts(马萨诸塞大学)

Haw-Shiuan Chang, Nikhil Agarwal, Andrew McCallum

AI总结:

本研究发现神经序列推荐器输出 softmax 层的单一隐藏状态与静态嵌入是处理重复项困难的根源,通过引入 softmax-CPR 替代架构使模型学会何时复制或排除输入项,在 12 个数据集上以同等模型规模显著提升了推荐性能。

摘要:

最近的研究表明,现有的神经模型在处理序列推荐任务中的重复项时存在困难。然而,我们对这种困难的理解仍然有限。在本研究中,我们通过识别问题的一个主要来源来大幅推进这一领域:输出 softmax 层中的单一隐藏状态嵌入和静态项嵌入。具体而言,softmax 层中全局项嵌入的相似性结构有时会迫使单一隐藏状态嵌入在复制是更好选择时接近新项,而有时又会不适当地迫使隐藏状态接近输入中的项。为了缓解这一问题,我们将最近提出的 softmax 替代方案(如 softmax-CPR)适配到序列推荐任务中,并证明新的 softmax 架构释放了神经编码器在学习何时复制以及何时排除输入序列中项的能力。仅通过对 SASRec 和 GRU4Rec 的输出 softmax 层进行一些简单修改,softmax-CPR 在 12 个数据集上取得了一致的改进。在模型大小几乎相同的情况下,我们的最佳方法不仅将 GRU4Rec 在 5 个包含重复项的数据集上的平均 NDCG@10 提高了 10%(单独为 4%-17%),还将 7 个不含重复项的数据集提高了 24%(8%-39%)!

英文摘要:

Recent studies suggest that the existing neural models have difficulty handling repeated items in sequential recommendation tasks. However, our understanding of this difficulty is still limited. In this study, we substantially advance this field by identifying a major source of the problem: the single hidden state embedding and static item embeddings in the output softmax layer. Specifically, the similarity structure of the global item embeddings in the softmax layer sometimes forces the single hidden state embedding to be close to new items when copying is a better choice, while sometimes forcing the hidden state to be close to the items from the input inappropriately. To alleviate the problem, we adapt the recently-proposed softmax alternatives such as softmax-CPR to sequential recommendation tasks and demonstrate that the new softmax architectures unleash the capability of the neural encoder on learning when to copy and when to exclude the items from the input sequence. By only making some simple modifications on the output softmax layer for SASRec and GRU4Rec, softmax-CPR achieves consistent improvement in 12 datasets. With almost the same model size, our best method not only improves the average NDCG@10 of GRU4Rec in 5 datasets with duplicated items by 10% (4%-17% individually) but also improves 7 datasets without duplicated items by 24% (8%-39%)!

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑