发表机构
University of Arkansas at Little Rock(阿肯色大学小石城分校)
机构由 AI 辅助整理,请以论文原文为准。AI 中文总结
NeuRoute是基于Logit引导的神经路由的轻量级ANN索引框架,通过训练神经网络编码器生成二进制地址,在十亿级向量搜索中实现亚小时级索引构建,且精度-吞吐量权衡优异。
AI 中文摘要
构建十亿级规模的近似最近邻(ANN)索引通常受限于昂贵的全局聚类或图构建,使得索引构建时间成为首要的系统问题。本文提出NeuRoute,一种基于学习的哈希索引,将短二进制码转化为适用于大规模向量搜索的有效路由原语。NeuRoute训练轻量级神经网络编码器,采用选择性相似性保持目标函数以生成均衡的二进制地址;构建索引时,NeuRoute通过向量的编码将其组织为桶,并在编码器的低维空间中执行桶内局部聚类以形成质心。查询阶段,NeuRoute将编码器logits作为不确定性信号:利用偏差-阈值分数对不确定位扰动进行优先级排序,以实现查询自适应多桶探测;通过桶内局部质心与查询的距离对其打分,形成紧凑的候选质心集;并采用基于堆质量的早停策略的质心阶段门控,在精确细化前修剪低价值质心。在十亿级基准测试中,NeuRoute实现了优异的精度-吞吐量权衡,且索引构建速度快:在BigANN-1B数据集上,其在2414 QPS时达到90.3%的Recall@10,在可比精度下比OPQ+IVF-PQ(细化)快1.7倍,且在BigANN-1B和Deep1B-1B上完成端到端训练+构建均耗时不足1小时。这些结果表明,基于Logit引导的神经路由可使哈希在十亿级规模上成为具有竞争力的轻量级ANN索引框架。源代码和相关成果可在该https URL获取。
英文摘要
Building approximate nearest neighbor (ANN) indexes at billion scale is often dominated by expensive global clustering or graph construction, making time-to-index a first-order systems concern. We present NeuRoute, a learned hashing index that turns short binary codes into an effective routing primitive for large-scale vector search. NeuRoute trains a lightweight neural network encoder with a selective similarity-preserving objective to produce well-balanced binary addresses. During construction, NeuRoute organizes vectors into buckets by their codes and performs bucket-local clustering in the encoder's low-dimensional space to form centroids. At query time, NeuRoute exploits the encoder logits as an uncertainty signal: it uses deviation-to-threshold scores to prioritize uncertain-bit perturbations for query-adaptive multi-bucket probing, scores bucket-local centroids by their distances to the query to form a compact candidate cluster set, and applies centroid-stage gating with heap-quality-driven early stopping to prune low-value clusters before exact refinement. On billion-scale benchmarks, NeuRoute achieves strong accuracy-throughput trade-offs with fast index construction: on BigANN-1B it reaches $90.3\%$ Recall@10 at 2,414 QPS and is $1.7\times$ faster than OPQ+IVF-PQ (refine) at comparable accuracy, while completing end-to-end training+construction in under an hour on both BigANN-1B and Deep1B-1B. These results show that logit-guided neural routing can make hashing competitive as a lightweight ANN indexing framework at billion scale. Source code and artifacts are available at https://github.com/XingqiaoWang/NeuRoute.
Comments34 pages, 9 figures