We present LiGR, a large-scale ranking framework developed at LinkedIn that brings state-of-the-art transformer-based modeling architectures into production. We introduce a modified transformer architecture that incorporates learned normalization and simultaneous set-wise attention to user history and ranked items. This architecture enables several breakthrough achievements, including: (1) the deprecation of most manually designed feature engineering, outperforming the prior state-of-the-art system using only few features (compared to hundreds in the baseline), (2) validation of the scaling law for ranking systems, showing improved performance with larger models, more training data, and longer context sequences, and (3) simultaneous joint scoring of items in a set-wise manner, leading to automated improvements in diversity. To enable efficient serving of large ranking models, we describe techniques to scale inference effectively using single-pass processing of user history and set-wise attention. We also summarize key insights from various ablation studies and A/B tests, highlighting the most impactful technical approaches.
Simple and Scalable Sparse k-means Clustering via Feature Ranking
Department of Statistical Science, Duke University（杜克大学统计科学系）；Departments of Computational Medicine, Statistics, and Human Genetics, UCLA（加州大学洛杉矶分校计算医学、统计与人类遗传学系）
Top-N-Rank: A Scalable List-wise Ranking Method for Recommender Systems
Pennsylvania State University（宾夕法尼亚州立大学）；South China University of Technology（华南理工大学）
我们提出了Top-N-Rank，一种新颖的列表式学习排名模型，用于可靠地推荐N个排名靠前的项目。所提出的模型优化了广泛使用的折扣累积增益（DCG）目标函数的变体，其在两个重要方面不同于DCG：（i）它将DCG的评估仅限于排名列表中的前N个项目，从而消除了低排名项目对学习的排名函数的影响;以及（ii）它结合了允许模型利用具有不同可靠性或可信度级别的多种类型的隐式反馈的权重。由于得到的目标函数是非光滑的，因此优化具有挑战性，我们考虑使用传统的S形函数和校正线性单元（ReLU）对目标函数进行两个光滑近似。我们提出了一个家庭的学习排名算法（Top-N-Rank），工作与任何光滑的目标函数。然后，引入了一个更有效的变体Top-N-Rank.ReLU，该变体有效地利用了ReLU函数的性质，将Top-N-Rank的计算复杂性从用户评分的平均项目数的二次降低到线性。我们使用两个广泛使用的基准测试，即MovieLens数据集和Amazon Video Games数据集的实验结果表明：（i）目标函数的“top-N截断”大大提高了前N个推荐的排名质量;（ii）与使用sigmoid相比，使用ReLU来平滑目标函数在排名质量和运行时间方面都有显着改善;以及（iii）Top-N-Rank.ReLU在排名质量方面大大优于表现良好的列表式排名方法。
