检索是推荐系统的初始阶段,其任务是从数千万候选物品构成的物品池中筛选出数千个物品。基于嵌入的检索(Embedding Based Retrieval,EBR)一直是解决该问题的典型选择,用于应对深度神经网络在海量物品语料库上的计算需求。EBR利用双塔网络或孪生网络学习用户和物品的表示,并采用近似最近邻(Approximate Nearest Neighbor,ANN)搜索高效检索相关物品。尽管EBR在工业界广受欢迎,但它仍面临局限。双塔架构依赖单一的点积交互,由于学习用户与物品之间富有表达力的交互的能力有限,难以捕捉复杂的数据分布。此外,ANN索引构建以及用户和物品的表示学习通常相互分离,由此导致的不一致会因表示漂移(例如持续在线训练)和物品漂移(例如物品过期以及新物品加入)而加剧。本文提出分层结构化神经网络(Hierarchical Structured Neural Network,HSNN),这是一种高效的深度神经网络模型,用于在检索任务中学习超越常用点积的复杂用户与物品交互,并实现相对于语料库规模的亚线性计算成本。本文设计了模块化神经网络(Modular Neural Network,MoNN),在保证效率的同时保持交互学习的高表达能力。多个MoNN构成的混合体在分层物品索引上运行,以实现广泛的计算共享,从而能够扩展到大规模语料库。MoNN与分层索引被联合学习,以持续适应用户兴趣和物品分布两方面的分布变化。与主流方法相比,HSNN在离线评估中取得了显著提升。
英文摘要:
Retrieval, the initial stage of a recommendation system, is tasked with down-selecting items from a pool of tens of millions of candidates to a few thousands. Embedding Based Retrieval (EBR) has been a typical choice for this problem, addressing the computational demands of deep neural networks across vast item corpora. EBR utilizes Two Tower or Siamese Networks to learn representations for users and items, and employ Approximate Nearest Neighbor (ANN) search to efficiently retrieve relevant items. Despite its popularity in industry, EBR faces limitations. The Two Tower architecture, relying on a single dot product interaction, struggles to capture complex data distributions due to limited capability in learning expressive interactions between users and items. Additionally, ANN index building and representation learning for user and item are often separate, leading to inconsistencies exacerbated by representation (e.g. continuous online training) and item drift (e.g. items expired and new items added). In this paper, we introduce the Hierarchical Structured Neural Network (HSNN), an efficient deep neural network model to learn intricate user and item interactions beyond the commonly used dot product in retrieval tasks, achieving sublinear computational costs relative to corpus size. A Modular Neural Network (MoNN) is designed to maintain high expressiveness for interaction learning while ensuring efficiency. A mixture of MoNNs operate on a hierarchical item index to achieve extensive computation sharing, enabling it to scale up to large corpus size. MoNN and the hierarchical index are jointly learnt to continuously adapt to distribution shifts in both user interests and item distributions. HSNN achieves substantial improvement in offline evaluation compared to prevailing methods.
Semantic Retrieval Augmented Contrastive Learning for Sequential Recommendation
City University of Hong Kong(香港城市大学);Huazhong University of Science and Technology(华中科技大学);Tencent(腾讯);Shenzhen Technology University(深圳技术大学);Peking University(北京大学);Shenzhen University(深圳大学)
DISCO: A Hierarchical Disentangled Cognitive Diagnosis Framework for Interpretable Job Recommendation
School of Artificial Intelligence, Anhui University(安徽大学人工智能学院);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);University of Science and Technology of China(中国科学技术大学);Institutes of Physical Science and Information Technology, Anhui University(安徽大学物理与信息技术研究院);School of Computer Science and Technology, Anhui University(安徽大学计算机科学与技术学院);Computer Network Information Center, Chinese Academy of Sciences(中国科学院计算机网络信息中心)
WARBERT: A Hierarchical BERT-based Model for Web API Recommendation
School of Software Engineering(软件工程学院);School of Computer Science and Technology(计算机科学与技术学院);Huazhong University of Science and Technology(华中科技大学);SmartX
Knowledge-Enhanced Hierarchical Graph Transformer Network for Multi-Behavior Recommendation
South China University of Technology(华南理工大学);JD Finance America Corporation(京东金融美国公司);Communication and Computer Network Laboratory of Guangdong(广东省通信与计算机网络实验室);Peng Cheng Laboratory(鹏城实验室);Simon Fraser University(西蒙弗雷泽大学)
RRCM: Ranking-Driven Retrieval over Collaborative and Meta Memories for LLM Recommendation
The University of Texas at Austin(德克萨斯大学奥斯汀分校);University of Illinois at Chicago(伊利诺伊大学香槟分校);Capital One AI Foundations(Capital One AI基金会);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校)