面向表格机器学习的智能体搜索空间

中文摘要 AI 辅助

尽管基于LLM的智能体在规划、代码生成和调试方面取得了快速进展，但它们在表格机器学习中的实际价值仍未得到充分探索。在本文中，我们研究了一个具体用例：最先进的智能体AI系统能否为成熟的表格模型设计扩展的HPO搜索空间，使其性能优于模型作者提供的标准搜索空间。具体来说，我们将每个表格模型表示为涵盖预处理、嵌入、架构、训练和推理的模块化流水线。然后，我们让智能体为每个模块提出候选代码实现，并使用经典HPO算法在候选实现与模型默认超参数上联合优化。与基础HPO空间相比，扩展搜索空间在45个数据集套件上几乎提升了每个模型族的性能，平均相对增益为0.6%，在中小型回归数据集上提升至2.0%。值得注意的是，这些增益无需额外调优成本：在相同调优和集成预算下，扩展空间优于基础空间。这些增益也迁移到了最近的TabArena基准上，其中智能体空间提升了五个模型族中四个的官方Elo分数，且两个最强的智能体集成超越了最佳AutoGluon常规模型集成。总体而言，我们的研究表明，LLM智能体通过扩展设计空间可以为表格机器学习提供实际价值。

英文摘要

Despite the rapid progress of LLM-based agents for planning, code generation, and debugging, their practical value for tabular machine learning remains underexplored. In this paper, we investigate a concrete use case: whether state-of-the-art agentic AI systems can design extended HPO search spaces for established tabular models that outperform the standard search spaces provided by the model authors. Specifically, we represent each tabular model as a modular pipeline covering preprocessing, embeddings, architecture, training, and inference. We then task the agent to propose candidate code implementations for each module and use a classical HPO algorithm to jointly optimize over these candidates and the model's default hyperparameters. Compared with the base HPO spaces, the expanded search spaces improve the performance of nearly every model family across a suite of 45 datasets, with average relative gains of 0.6%, rising to 2.0% on small-to-medium regression datasets. Notably, these gains come at no extra tuning cost: the enlarged spaces outperform the base under the same tuning and ensembling budgets. The gains transfer to the recent TabArena benchmark, where the agentic spaces improve the official Elo scores of four of the five model families and the two strongest agentic ensembles surpass the best AutoGluon ensemble of conventional models. Overall, our study suggests that LLM agents can provide practical value for tabular ML by expanding the design space.

发表机构

• Yandex
• HSE University（高等经济大学）

机构由 AI 辅助整理，请以论文原文为准。