流式分层推断与表格基础模型
Streaming Hierarchical Inference with Tabular Foundation Models
- Polytechnic of Porto（波尔图理工学院）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
针对表格基础模型在数据流中部署的通信与延迟问题，提出HINT分层推断框架，结合边缘检索与云端推断，通过卸载阈值和邻域策略平衡性能与成本，实验验证其有效性。
AI中文摘要：
表格基础模型（TFMs）近期通过上下文学习展示了强大的预测性能，但其在高吞吐量数据流中的部署仍因通信开销和延迟而面临挑战。我们提出HINT，一种分层推断框架，将边缘端检索与云端TFM推断相结合。在滑动窗口上维护的基于图的近似最近邻存储器提供局部预测和不确定性估计，使得置信样本可在本地处理，而不确定实例则被选择性卸载，连同其检索到的上下文，一起发送至云端托管的TFM。该框架暴露了一个卸载阈值和邻域检索策略，可调整以平衡预测性能与通信成本。实验表明，HINT始终能识别出有利的权衡。
英文摘要：
Tabular Foundation Models (TFMs) have recently demonstrated strong predictive performance through in-context learning, but their deployment in high-throughput data streams remains challenging due to communication overhead and latency. We propose \textit{HINT}, a hierarchical inference framework that combines edge-based retrieval with cloud-based TFM inference. A graph-based approximate nearest neighbor memory maintained over a sliding window provides local predictions and uncertainty estimates, allowing confident samples to be processed locally while uncertain instances are selectively offloaded, together with their retrieved context, to a cloud-hosted TFM. The framework exposes an offloading threshold and a neighborhood retrieval policy that can be varied to balance predictive performance and communication cost. Experiments show \textit{HINT} consistently identifies favorable trade-offs.
