如何训练你的过滤器:你应该学习、堆叠还是适应?
How to Train Your Filter: Should You Learn, Stack or Adapt?
AI总结:
本文首次全面评估了学习、堆叠和自适应过滤器,揭示了不同方法在假阳性率、鲁棒性和性能上的权衡,为不同应用场景提供了选择依据。
摘要:
过滤器在计算机科学中无处不在,能够实现空间高效的近似成员测试。自从1970年布洛姆过滤器被提出以来,数十年的研究不断改进其空间效率和性能。最近,三种新的范式出现了,通过利用输入集合之外的信息,实现了假阳性率(FPRs)的量级提升:(1)学习过滤器训练模型来区分(非)成员;(2)堆叠过滤器利用负工作负载样本构建级联层;(3)自适应过滤器根据假阳性反馈更新内部表示。然而,每种范式都针对特定的应用场景,引入了复杂的配置调优,并且在孤立状态下进行评估。这导致了不清楚的权衡和对这些方法如何比较以及何时最适用的理解空白。本文首次对学习、堆叠和自适应过滤器进行了全面评估,覆盖真实世界的数据集和查询工作负载。我们的结果揭示了关键的权衡:(1)学习过滤器可以达到10^2倍更低的假阳性率,但表现出高方差且在倾斜或动态工作负载下缺乏鲁棒性。关键的是,模型推理开销导致查询延迟比堆叠或自适应过滤器慢10^4倍。(2)堆叠过滤器在倾斜的工作负载上可靠地实现了10^3倍更低的假阳性率,但需要工作负载知识。(3)自适应过滤器在各种设置中都具有鲁棒性,能够在对抗性查询下实现10^3倍更低的假阳性率,而无需工作负载假设。基于我们的分析,学习过滤器适用于稳定的工作负载,其中输入特征能够有效训练模型且空间约束至关重要,堆叠过滤器在已知可靠查询分布时表现最佳,而自适应过滤器则最为通用,即使在动态或对抗性环境中也能提供理论上有界保证。
英文摘要:
Filters are ubiquitous in computer science, enabling space-efficient approximate membership testing. Since Bloom filters were introduced in 1970, decades of work improved their space efficiency and performance. Recently, three new paradigms have emerged offering orders-of-magnitude improvements in false positive rates (FPRs) by using information beyond the input set: (1) learned filters train a model to distinguish (non)members, (2) stacked filters use negative workload samples to build cascading layers, and (3) adaptive filters update internal representation in response to false positive feedback. Yet each paradigm targets specific use cases, introduces complex configuration tuning, and has been evaluated in isolation. This results in unclear trade-offs and a gap in understanding how these approaches compare and when each is most appropriate. This paper presents the first comprehensive evaluation of learned, stacked, and adaptive filters across real-world datasets and query workloads. Our results reveal critical trade-offs: (1) Learned filters achieve up to 10^2 times lower FPRs but exhibit high variance and lack robustness under skewed or dynamic workloads. Critically, model inference overhead leads to up to 10^4 times slower query latencies than stacked or adaptive filters. (2) Stacked filters reliably achieve up to 10^3 times lower FPRs on skewed workloads but require workload knowledge. (3) Adaptive filters are robust across settings, achieving up to 10^3 times lower FPRs under adversarial queries without workload assumptions. Based on our analysis, learned filters suit stable workloads where input features enable effective model training and space constraints are paramount, stacked filters excel when reliable query distributions are known, and adaptive filters are most generalizable, providing robust theoretically bound guarantees even in dynamic or adversarial environments.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。