arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2407.00734cs.DS
收藏

平衡学习排序:一种用于快速且均衡项目分桶的新型学习模型

Balanced Learned Sort: a new learned model for fast and balanced item bucketing

Paolo Ferragina, Mattia Odorisio

AI总结:

本文提出平衡学习排序模型,通过改进两层RMI并设计新算法结构实现快速均衡分桶,在33个数据集中的31个上超越6种对比排序器。

摘要:

本文旨在通过两个主要研究步骤,更好地理解在顺序排序数值数据中采用基于学习的方法的优势与局限。首先,我们研究了用于基于分布排序的不同学习模型,从一些已知模型(即两层RMI或简单线性模型)出发,然后引入一些新模型,这些模型要么改进了两层RMI,要么在其算法结构上完全新颖,从而具有空间高效、单调性以及非常快速地构建均衡桶的特点。我们在11个合成数据集上测试了这些模型,这些数据集来自2亿个64位浮点项的不同分布,从而得出关于它们最终性能以及在设计排序算法中有用性的提示。基于这些发现,我们从上述模型中选出最佳模型,将其嵌入一个新的基于学习的算法方案中,并设计出三种新的排序器。我们将在33个数据集(11个合成数据集和22个真实数据集)上对这些排序器与其他6种顺序排序器(5种经典排序器和1种学习型排序器,包括已知和新提出的)进行测试,数据集规模最高可达8亿项。我们的实验数据将表明,我们的学习型排序器在全部33个数据集(合成和真实)中的31个上取得了优越性能。总之,这些实验结果一方面为主要问题提供了全面的答案:哪种用于基于分布排序的算法结构适合利用学习模型以实现高效性能?另一方面,它们也留下了若干其他研究和工程问题,涉及设计一种在不同输入分布上具有鲁棒性的高性能顺序排序器。

英文摘要:

This paper aims to better understand the strengths and limitations of adopting learned-based approaches in sequential sorting numerical data, via two main research steps. First, we study different learned models for distribution-based sorting, starting from some known ones (i.e., two-layer RMI or simple linear models) and then introducing some novel models that either improve the two-layer RMI or are fully new in their algorithmic structure thus resulting space efficient, monotonic, and very fast in building balanced buckets. We test those models over 11 synthetic datasets drawn from different distributions of 200M 64-bit floating-point items, so deriving hints about their ultimate performance and usefulness in designing a sorting algorithm. Based on these findings, we select and plug the best models from above in a new learned-based algorithmic scheme and devise three new sorters that we will test against other 6 sequential sorters (5 classic and 1 learned, known and new ones) over 33 datasets (11 synthetic and 22 real), whose size will be up to 800M items. Our experimental figures will show that our learned sorters achieve superior performance on 31 out of all 33 datasets (synthetic and real). In conclusion, these experimental results provide, on the one hand, a comprehensive answer to the main question: Which algorithmic structure for distribution-based sorting is suited to leverage a learned model in order to achieve efficient performance? and, on the other hand, they leave open several other research and engineering questions about the design of a highly performing sequential sorter that is robust over different input distributions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑