The Anxiety of Influence: Bloom Filters in Transformer Attention Heads
影响的焦虑:变换器注意力头中的布洛姆过滤器
专题命中 其他LLM :language model(abstract);分类 cs.CL、cs.AI、cs.LG
AI总结 研究发现变换器注意力头中存在布洛姆过滤器功能,通过成员测试策略实现高效重复标记识别,且误报率随嵌入距离降低而减少。
Comments 13 pages, 8 figures, code at https://github.com/pbalogh/anxiety-of-influence v2: L3H0 reclassified as prefix-attention head following confound control. Capacity analysis updated. Duplicate-token head overlap experiment added v3: All experiments were independently validated on CPU to rule out hardware-specific computation artifacts. Results are consistent across backends