AI 中文总结
针对表格上下文学习的准确率与吞吐量权衡问题，提出QCOC方法，通过编译上下文示例的KV缓存为共享原型实现高效推理，在保持高准确率的同时大幅提升速度与压缩比。
AI 中文摘要
表格上下文学习（ICL）已成为一种无需训练且准确的表格预测范式，但当前对其上下文示例进行压缩的方法存在准确率与吞吐量的权衡问题：固定子集可能牺牲准确率，而针对特定查询的检索则会限制跨查询的缓存复用与批处理，从而降低吞吐量。我们提出QCOC（Query-Calibrated Operator Compression，查询校准算子压缩），该方法通过将上下文示例的完整键值（KV）缓存一次性编译为紧凑内存，供后续所有查询共享，以此利用上下文示例的可交换性与重复使用特性。QCOC不保留原始示例，而是将其状态聚类为联合KV原型，保留每个聚类的多重性及原始示例数量，并通过锚定闭式解针对上下文示例生成的注意力查询向量校准原型值。原型压缩带来加速，而值拟合则有助于保留准确率。在64个保留的OpenML-CC18数据集上，QCOC在两种保留数量下均达到了所对比的压缩与检索方法中的最高平均准确率；在7个长表格的12种配置中，它在10种配置下排名压缩方法第一，平均仅比完整上下文低0.23个百分点。将8192个上下文示例压缩至512个内存槽，可实现10.5倍的缓存压缩率；排除一次性编译时间，在单核CPU上针对1000个查询的在线服务对比中，QCOC比动态检索基线快达508倍，比完整上下文推理快1.98倍。这些结果表明，QCOC可实现跨查询的紧凑内存复用与高效推理，同时保留接近完整上下文的准确率。
英文摘要
Tabular in-context learning (ICL) has emerged as a training-free and accurate paradigm for tabular prediction, but current approaches to compressing its in-context examples face an accuracy-throughput tradeoff: fixed subsets can sacrifice accuracy, while query-specific retrieval limits cache reuse and batching across queries, reducing throughput. We propose QCOC (Query-Calibrated Operator Compression), which exploits the exchangeability and repeated use of in-context examples by compiling their full KV cache once into compact memory shared across subsequent queries. Instead of retaining raw examples, QCOC clusters their states into joint-KV prototypes, preserves per-cluster multiplicities and the original example count, and calibrates prototype values against attention query vectors produced by the in-context examples through an anchored closed-form solution. Prototype compression drives the speedup, while value fitting helps preserve accuracy. On 64 held-out OpenML-CC18 datasets, QCOC achieves the highest mean accuracy among the compared compression and retrieval methods at both retained counts. Across 12 configurations on seven long tables, it ranks first among compressed methods in ten and averages 0.23 percentage points below full context. Compressing 8,192 in-context examples to 512 memory slots yields a 10.5x cache compression ratio; excluding one-time compilation, in a single-core CPU online-serving comparison over 1,000 queries, QCOC is up to 508x faster than dynamic retrieval baselines and 1.98x faster than full-context inference. These results show that QCOC enables compact-memory reuse and efficient inference across queries while retaining accuracy close to full context.
Comments24 pages, 7 figures. Includes supplementary material
