Vector databases have become a cornerstone of modern information retrieval, powering applications in recommendation, search, and retrieval-augmented generation (RAG) pipelines. However, scaling approximate nearest neighbor (ANN) search to high recall under strict latency SLOs remains fundamentally constrained by memory capacity and I/O bandwidth. Disk-based vector search systems suffer severe latency degradation at high accuracy, while fully in-memory solutions incur prohibitive memory costs at billion-scale. Despite the central role of caching in traditional databases, vector search lacks a general query-level caching layer capable of amortizing repeated query work.
We present QVCache, the first backend-agnostic, query-level caching system for ANN search with bounded memory footprint. QVCache exploits semantic query repetition by performing similarity-aware caching rather than exact-match lookup. It dynamically learns region-specific distance thresholds using an online learning algorithm, enabling recall-preserving cache hits while bounding lookup latency and memory usage independently of dataset size. QVCache operates as a drop-in layer for existing vector databases. It maintains a megabyte-scale memory footprint and achieves sub-millisecond cache-hit latency, reducing end-to-end query latency by up to 40-1000x when integrated with existing ANN systems. For workloads exhibiting temporal-semantic locality, QVCache substantially reduces latency while preserving recall comparable to the underlying ANN backend, establishing it as a missing but essential caching layer for scalable vector search.
QCFuse: Query-Aware Cache Fusion via Compressed View for Efficient RAG Serving
Zhejiang University(浙江大学);East China Normal University(华东师范大学);Ant Group(蚂蚁集团);The Hong Kong Polytechnic University(香港理工大学);Zhejiang Normal University(浙江师范大学);Tongji University(同济大学);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));The Hong Kong University of Science and Technology (Guangzhou)(香港科学与技术大学(广州));The Hong Kong University of Science and Technology(香港科学与技术大学)
Metadata Caching in Presto: Towards Fast Data Processing
Presto是一个用于OLAP的开源分布式SQL查询引擎,旨在实现“SQL on everything”。自2013年开源以来,Presto一直在大规模数据分析中受到欢迎,并吸引了众多企业的采用。从Presto的开发和运行来看,我们目睹了Presto工作节点中解析面向列的数据文件的大量CPU消耗。这阻碍了包括Meta在内的一些公司增加分析数据量。在本文中,我们提出了一个元数据缓存层,建立在Alluxio SDK缓存之上,并合并在每个Presto工作节点中,以缓存文件解析中的中间结果。元数据缓存提供了两种缓存方法:缓存来自原始数据文件的解压缩元数据字节和缓存已格式化的元数据对象。我们对Presto上的TPC-DS基准测试的评估表明,当缓存是热的时,第一种方法可以将查询的CPU消耗减少10%-20%,而第二种方法可以将CPU使用量减少20%-40%。
FIER: Fine-Grained and Efficient KV Cache Retrieval for Long-context LLM Inference
The University of Arizona(亚利桑那大学);The University of North Carolina at Chapel Hill(北卡罗来纳大学教堂山分校);The University of Virginia(弗吉尼亚大学);University of Pittsburgh(匹兹堡大学)