熵穿孔布隆过滤器用于内存高效的机器学习
Entropy-Punctured Bloom Filters for Memory-Efficient Machine Learning
浏览论文内容
中文总结 AI 辅助
本文提出熵穿孔布隆过滤器，通过经验熵移除低变异性位位置来压缩特征表示，在保持预测结构的同时提高内存效率，实验表明其在回归任务中优于经典压缩方法。
中文摘要 AI 辅助
在存储、传输成本、带宽或隐私约束限制访问原始数据的机器学习场景中，内存高效的特征表示日益重要。布隆过滤器（BF）编码为工程化特征提供了紧凑的概率表示，但它们在结构压缩下的行为及其在回归任务中的适用性仍未得到充分探索。在这项工作中，我们提出了熵穿孔布隆过滤器，这是一种内存感知的编码策略，它利用经验熵识别并移除低变异性的位位置。从量化特征的固定长度BF编码出发，所提出的方法生成缩减的表示，这些表示保留了预测结构，同时相对于编码表示大小提高了预测效率。我们在多种回归数据集上评估了该方法，在无泄漏评估协议和近似匹配的表示大小下，比较了原始特征、主成分分析（PCA）、随机投影（RP）和布隆过滤器变体。性能使用岭回归、XGBoost和神经网络进行评估，预测效率以相对于每个样本编码表示大小的R2来衡量。结果表明，布隆过滤器编码在实现显著存储节省的同时，仍能与经典压缩表示保持竞争力。基于熵的穿孔进一步减小了表示大小，且预测保真度损失极小，从而提高了预测效率。这些发现表明，熵穿孔布隆过滤器为内存受限的机器学习提供了一种有效的表示级压缩方法。
英文摘要
Memory-efficient feature representations are increasingly important in machine learning settings where storage, transmission cost, bandwidth, or privacy constraints limit access to raw data. Bloom Filter (BF) encodings provide compact probabilistic representations of engineered features, but their behavior under structural compression and their applicability to regression tasks remain underexplored. In this work, we propose entropy-punctured Bloom Filters, a memory-aware encoding strategy that removes low-variability bit positions identified using empirical entropy. Starting from fixed-length BF encodings of quantized features, the proposed approach produces reduced representations that preserve predictive structure while improving predictive efficiency relative to encoded representation size. We evaluate the approach on diverse regression datasets, comparing raw features, Principal Component Analysis (PCA), Random Projection (RP), and Bloom Filter variants under leakage-free evaluation protocols and approximately matched representation sizes. Performance is assessed using ridge regression, XGBoost, and neural networks, with predictive efficiency measured as R2 relative to encoded representation size per sample. Results show that Bloom Filter encodings remain competitive with classical compressed representations while achieving substantial storage savings. Entropy-based puncturing further reduces representation size with minimal loss in predictive fidelity, yielding improved predictive efficiency. These findings demonstrate that entropy-punctured Bloom Filters provide an effective representation-level compression approach for memory-constrained machine learning.
发表机构
- Florida Atlantic University（佛罗里达大西洋大学）
机构由 AI 辅助整理，请以论文原文为准。
