arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2408.06653cs.IRcs.AI
收藏

分层结构化神经网络:面向大规模推荐的高效检索扩展

Hierarchical Structured Neural Network: Efficient Retrieval Scaling for Large Scale Recommendation

Meta Platforms Inc.(元平台公司)
Kaushik Rangadurai, Siyang Yuan, Minhui Huang, Yiqun Liu, Golnaz Ghasemiesfeh, Yunchen Pu, Haiyu Lu, Xingfeng He, Fangzhou Xu, Andrew Cui, Vidhoon Viswanathan, … 展开作者

Kaushik Rangadurai, Siyang Yuan, Minhui Huang, Yiqun Liu, Golnaz Ghasemiesfeh, Yunchen Pu, Haiyu Lu, Xingfeng He, Fangzhou Xu, Andrew Cui, Vidhoon Viswanathan, Lin Yang, Liang Wang, Jiyan Yang, Chonglin Sun

AI总结:

针对EBR双塔点积表达不足及索引与表示学习分离的问题,提出HSNN,通过MoNN与分层物品索引联合学习实现复杂交互建模、计算共享和亚线性扩展,离线效果显著提升。

摘要:

检索是推荐系统的初始阶段,其任务是从数千万候选物品构成的物品池中筛选出数千个物品。基于嵌入的检索(Embedding Based Retrieval,EBR)一直是解决该问题的典型选择,用于应对深度神经网络在海量物品语料库上的计算需求。EBR利用双塔网络或孪生网络学习用户和物品的表示,并采用近似最近邻(Approximate Nearest Neighbor,ANN)搜索高效检索相关物品。尽管EBR在工业界广受欢迎,但它仍面临局限。双塔架构依赖单一的点积交互,由于学习用户与物品之间富有表达力的交互的能力有限,难以捕捉复杂的数据分布。此外,ANN索引构建以及用户和物品的表示学习通常相互分离,由此导致的不一致会因表示漂移(例如持续在线训练)和物品漂移(例如物品过期以及新物品加入)而加剧。本文提出分层结构化神经网络(Hierarchical Structured Neural Network,HSNN),这是一种高效的深度神经网络模型,用于在检索任务中学习超越常用点积的复杂用户与物品交互,并实现相对于语料库规模的亚线性计算成本。本文设计了模块化神经网络(Modular Neural Network,MoNN),在保证效率的同时保持交互学习的高表达能力。多个MoNN构成的混合体在分层物品索引上运行,以实现广泛的计算共享,从而能够扩展到大规模语料库。MoNN与分层索引被联合学习,以持续适应用户兴趣和物品分布两方面的分布变化。与主流方法相比,HSNN在离线评估中取得了显著提升。

英文摘要:

Retrieval, the initial stage of a recommendation system, is tasked with down-selecting items from a pool of tens of millions of candidates to a few thousands. Embedding Based Retrieval (EBR) has been a typical choice for this problem, addressing the computational demands of deep neural networks across vast item corpora. EBR utilizes Two Tower or Siamese Networks to learn representations for users and items, and employ Approximate Nearest Neighbor (ANN) search to efficiently retrieve relevant items. Despite its popularity in industry, EBR faces limitations. The Two Tower architecture, relying on a single dot product interaction, struggles to capture complex data distributions due to limited capability in learning expressive interactions between users and items. Additionally, ANN index building and representation learning for user and item are often separate, leading to inconsistencies exacerbated by representation (e.g. continuous online training) and item drift (e.g. items expired and new items added). In this paper, we introduce the Hierarchical Structured Neural Network (HSNN), an efficient deep neural network model to learn intricate user and item interactions beyond the commonly used dot product in retrieval tasks, achieving sublinear computational costs relative to corpus size. A Modular Neural Network (MoNN) is designed to maintain high expressiveness for interaction learning while ensuring efficiency. A mixture of MoNNs operate on a hierarchical item index to achieve extensive computation sharing, enabling it to scale up to large corpus size. MoNN and the hierarchical index are jointly learnt to continuously adapt to distribution shifts in both user interests and item distributions. HSNN achieves substantial improvement in offline evaluation compared to prevailing methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑