表格化上下文学习的早停
Early Stopping Tabular In-Context Learning
AI总结：
针对表格基础模型上下文学习推理成本高的问题，提出在每个Transformer编码器层后动态评估并早停的机制，结合预训练逐层解码器，在保持预测性能的同时显著提升推理速度。
摘要：
表格基础模型通过上下文学习在各种表格学习任务中表现出色，无需任何下游微调即可提供强大的泛化能力。然而，它们的推理时间成本仍然很高，特别是对于较大的数据集。为了解决这个问题，我们提出对上下文学习过程进行早停。我们通过在每个 Transformer 编码器层之后动态评估是否停止上下文学习来实现这一点。一旦停止，我们就使用预训练的逐层解码器对嵌入进行解码。在34个小规模分类任务上的实验表明，早停上下文学习可使推理加速高达1.3倍，且预测性能的下降微乎其微。为了评估可扩展性，我们进一步在五个较大规模分类任务上评估了我们的方法，实现了高达2.2倍的加速。我们的结果证明了提前退出作为提高表格上下文学习效率的有效且实用策略的潜力。
英文摘要：
Tabular foundation models have shown strong performance across various tabular learning tasks via in-context learning, offering robust generalization without any downstream finetuning. However, their inference-time costs remain high, particularly for larger datasets. To address this, we propose early-stopping the in-context learning process. We achieve this by dynamically evaluating whether to stop in-context learning after each Transformer encoder layer. Once stopped, we decode the embedding using a pre-trained layer-wise decoder. Experiments across 34 small classification tasks size show that early stopping in-context learning accelerates inference by up to x1.3 with negligible degradation in predictive performance. To assess scalability, we further evaluate our method on five larger classification tasks, achieving speedups of up to x2.2. Our results demonstrate the potential of early exiting as an effective and practical strategy for improving the efficiency of tabular in-context learning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
