PT-Ranking：神经学习排序的基准测试平台
PT-Ranking: A Benchmarking Platform for Neural Learning-to-Rank
摘要：
深度神经网络已成为研究学习排序算法方面的研究人员的首选。不幸的是，要找到实现最佳排名性能的超参数的最佳设置并不容易。因此，开发新模型并与现有方法进行公平比较变得越来越困难，特别是对于新手。在这项工作中，我们提出了PT-Ranking，这是一个基于PyTorch的开源项目，用于开发和评估使用深度神经网络作为基础来构建评分函数的学习排序方法。一方面，PT排序包括许多有代表性的学习排序方法。除了传统的优化框架通过经验风险最小化，对抗优化框架也被集成。此外，PT排名的模块化设计提供了一套构建模块，用户可以利用这些模块来开发新的排名模型。另一方面，PT排名支持基于广泛使用的数据集（例如，Yahoo！LETOR和Istella LETOR）在不同的度量方面，例如精度、MAP、nDCG、nERR。通过以指定的比率随机掩蔽地面实况标签，PT排名允许检查未标记的查询-文档对的比率在多大程度上影响不同学习排序方法的性能。我们进一步进行了一系列演示实验，以清楚地显示不同因素对神经学习排序方法的影响，如激活函数，层数和优化策略。
英文摘要：
Deep neural networks has become the first choice for researchers working on algorithmic aspects of learning-to-rank. Unfortunately, it is not trivial to find the optimal setting of hyper-parameters that achieves the best ranking performance. As a result, it becomes more and more difficult to develop a new model and conduct a fair comparison with prior methods, especially for newcomers. In this work, we propose PT-Ranking, an open-source project based on PyTorch for developing and evaluating learning-to-rank methods using deep neural networks as the basis to construct a scoring function. On one hand, PT-Ranking includes many representative learning-to-rank methods. Besides the traditional optimization framework via empirical risk minimization, adversarial optimization framework is also integrated. Furthermore, PT-Ranking's modular design provides a set of building blocks that users can leverage to develop new ranking models. On the other hand, PT-Ranking supports to compare different learning-to-rank methods based on the widely used datasets (e.g., MSLR-WEB30K, Yahoo!LETOR and Istella LETOR) in terms of different metrics, such as precision, MAP, nDCG, nERR. By randomly masking the ground-truth labels with a specified ratio, PT-Ranking allows to examine to what extent the ratio of unlabelled query-document pairs affects the performance of different learning-to-rank methods. We further conducted a series of demo experiments to clearly show the effect of different factors on neural learning-to-rank methods, such as the activation function, the number of layers and the optimization strategy.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
