arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2607.01548cs.LGcs.AI
收藏

结构化数据的进化特征工程

Evolutionary Feature Engineering for Structured Data

University of Michigan(密歇根大学);Google Research(谷歌研究院)

Ege Onur Taga, Yilin Zhuang, M. Emrullah Ildiz, Petros Mol, Abhimanyu Das, Karthik Duraisamy, Samet Oymak

AI总结:

提出基于LLM进化的特征工程框架EFE,通过Python程序表示变换,利用数据集上下文和验证集性能反馈优化,在时间序列和表格预测任务中提升准确性和可解释性。

摘要:

大型语言模型越来越多地被用作进化优化中的开放式搜索算子。我们引入了进化特征工程(EFE),这是一个利用基于LLM的进化来发现结构化数据预处理变换的框架。EFE将变换表示为具有标准化fit/transform接口的Python程序,允许它们直接插入现有的机器学习流水线中。在进化过程中,候选程序使用数据集上下文、汇总统计和验证集上的下游性能反馈进行优化。我们在两个设置中实例化EFE。对于时间序列预测,EFE-Time学习可逆的、数据集特定的归一化,改进了现成的时间序列基础模型。它在数据集上平均将预测误差(MASE、WQL、MAE)降低3%或更多,在COVID-Deaths数据集上改进高达19%。值得注意的是,这些改进发生在最近的TSFM(如Chronos-2)上。对于表格预测,EFE-Tab进化出紧凑的特征程序,添加有用的可解释特征并移除冗余特征,改进或匹配现有的基于LLM的特征工程方法。我们发现EFE-Tab在经典决策树上特别有效,其中少量进化特征在保持可解释性的同时产生具有竞争力的准确性。总体而言,EFE证明了基于LLM的进化可以在自动处理结构化数据时提高准确性和可解释性。

英文摘要:

Large language models are increasingly used as open-ended search operators in evolutionary optimization. We introduce Evolutionary Feature Engineering (EFE), a framework for using LLM-based evolution to discover preprocessing transformations for structured data. EFE represents transformations as Python programs with a standardized fit/transform interface, allowing them to be inserted directly into existing machine learning pipelines. During evolution, candidate programs are refined using dataset context, summary statistics, and downstream performance feedback on validation set. We instantiate EFE in two settings. For time-series forecasting, EFE-Time learns invertible, dataset-specific normalizations that improve off-the-shelf time-series foundation models. It reduces forecasting errors (MASE, WQL, MAE) 3% or more when averaged across datasets and improvements are as much as 19% on the COVID-Deaths dataset. Notably, these improvements occur with recent TSFMs such as Chronos-2. For tabular prediction, EFE-Tab evolves compact feature programs that add useful interpretable features and remove redundant ones, improving or matching existing LLM-based feature-engineering methods. We found EFE-Tab to be particularly effective on classical decision trees, where small sets of evolved features yield competitive accuracy while preserving interpretability. Overall, EFE demonstrates that LLM-based evolution can improve both accuracy and interpretability when automatically tackling structured data.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑