AI 中文总结
本文提出Tydra混合架构，平衡了表格基础模型的性能与效率，在30个OpenML数据集上较TabPFN提速30%，且优于规模大10倍的Hydra模型，为表格基础模型提供了新方向。
AI 中文摘要
基于Transformer的表格基础模型（如TabPFN）具备强大的预测性能，但会随上下文长度产生二次计算成本；而基于亚二次复杂度状态空间模型（SSM）的替代方案（如Hydra）则以精度为代价换取效率。为平衡二者，本文提出Tydra，一种用于表格上下文学习的Transformer-SSM混合架构，其交替使用注意力层与SSM层。在30个OpenML数据集上，Tydra相比TabPFN减少了30%的推理时间，同时保留了其大部分预测性能；Tydra还优于规模约大10倍的Hydra模型，且推理速度更快。这些结果表明，混合架构是表格基础模型的一个有前景的研究方向。
英文摘要
Transformer-based tabular foundation models such as TabPFN achieve strong predictive performance but incur quadratic computational cost with context length. On the other hand, subquadratic SSM-based alternatives such as Hydra trade away accuracy for efficiency. To balance both, we introduce Tydra, a hybrid Transformer-State Space Model (SSM) architecture for tabular in-context learning that interleaves attention and SSM layers. Across 30 OpenML datasets, Tydra reduces inference time by 30% relative to TabPFN while retaining much of its predictive performance. Tydra also outperforms an approximately ten-times-larger Hydra model while providing faster inference. The results indicate that hybrid architectures are a promising direction for tabular foundation models.
