ANNLib:一种高效近似最近邻搜索的开发框架
ANNLib: A Development Framework for Efficient Approximate Nearest Neighbor Search
- UC Riverside(加州大学河滨分校)
- W&M(威廉与玛丽学院)
机构由 AI 辅助整理,请以论文原文为准。
AI总结:
针对现代深度学习中近似最近邻搜索难以兼顾功能与性能的问题,提出ANNLib框架,基于图算法解耦并优化算法和数据结构组件,集成多种算法和数据结构,为应用提供简单接口且性能更优。
AI中文摘要:
近似最近邻搜索(ANNS)在现代深度学习管道中起着关键作用。近期虽有许多ANNS系统被提出,但难以在最小编程工作量下兼顾广泛功能与高性能。我们提出ANNLib来填补这一空白。它基于流行的基于图的ANNS算法,为ANNS系统提供实现高性能和灵活功能的编程框架。我们仔细解耦并独立优化ANNS系统的算法和数据结构组件,还将先进算法和数据结构作为模块集成到ANNLib中。用户能选择组件组合实现复杂设置。实验表明,新方案为各种应用提供简单接口,性能与之前工作相当甚至更优。
英文摘要:
Approximate Nearest Neighbor Search (ANNS) plays a pivotal role in modern deep learning pipelines. Recently, many ANNS systems have been proposed to provide broad, flexible functionalities or achieve high performance. However, it is inherently difficult to achieve both. We propose ANNLib to address this gap. ANNLib is a library that provides a programming framework to achieve high performance and flexible functionalities for ANNS systems, based on popular graph-based ANNS algorithms. We carefully decouple and independently optimize both the algorithm and the data structure components in an ANNS system. In addition, we integrate state-of-the-art algorithms and data structures as modules in ANNLib, as well as our new designs. Users can choose combinations of components to support sophisticated settings with high performance, such as filtered search, fully dynamic updates, historical queries on snapshots, and range searches. Our experiments show that our new solution provides a simple interface for various applications, and achieves comparable or even better performance to previous work specifically for each application.