arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2604.12965cs.IR
收藏

基于大规模推荐的高效检索扩展与分层索引

Efficient Retrieval Scaling with Hierarchical Indexing for Large Scale Recommendation

Meta
Dongqi Fu, Kaushik Rangadurai, Haiyu Lu, Yunchen Pu, Siyang Yuan, Minhui Huang, Yiqun Liu, Golnaz Ghasemiesfeh, Xingfeng He, Fangzhou Xu, Andrew Cui, Vidhoon Vi… 展开作者

Dongqi Fu, Kaushik Rangadurai, Haiyu Lu, Yunchen Pu, Siyang Yuan, Minhui Huang, Yiqun Liu, Golnaz Ghasemiesfeh, Xingfeng He, Fangzhou Xu, Andrew Cui, Vidhoon Viswanathan, Lin Yang, Liang Wang, Jiyan Yang, Chonglin Sun

AI总结:

本文提出通过交叉注意力和残差量化联合学习分层索引,以提升大规模检索模型的效率和精确度,并在Meta实现实时广告推荐。

摘要:

随着数据量、计算资源和模型参数的增加,大量工业级检索模型被开发用于推荐任务。然而,有效部署这些大规模基础检索模型仍是一个关键挑战。本文探讨是否能学习基础检索模型内存中的分层结构,以减少检索成本同时保持精确度。为此,我们提出使用交叉注意力和残差量化联合学习分层索引,并在Meta部署,支持数十亿Facebook和Instagram用户的每日广告推荐。有趣的是,我们发现学习的索引中间节点对应一小部分高质量数据。在该数据集上微调模型进一步提升了推理性能,并具体化了推荐系统领域内的

英文摘要:

The increase in data volume, computational resources, and model parameters during training has led to the development of numerous large-scale industrial retrieval models for recommendation tasks. However, effectively and efficiently deploying these large-scale foundational retrieval models remains a critical challenge that has not been fully addressed. Common quick-win solutions for deploying these massive models include relying on offline computations (such as cached user dictionaries) or distilling large models into smaller ones. Yet, both approaches fall short of fully leveraging the representational and inference capabilities of foundational models. In this paper, we explore whether it is possible to learn a hierarchical organization over the memory of foundational retrieval models. Such a hierarchical structure would enable more efficient search by reducing retrieval costs while preserving exactness. To achieve this, we propose jointly learning a hierarchical index using cross-attention and residual quantization for large-scale retrieval models. We also present its real-world deployment at Meta, supporting daily advertisement recommendations for billions of Facebook and Instagram users. Interestingly, we discovered that the intermediate nodes in the learned index correspond to a small set of high-quality data. Fine-tuning the model on this set further improves inference performance, and concretize the concept of "test-time training" within the recommendation system domain. We demonstrate these findings using both internal and public datasets with strong baseline comparisons and hope they contribute to the community's efforts in developing the next generation of foundational retrieval models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑