FAMOSE：一种用于自动化特征发现的ReAct方法
FAMOSE: A ReAct Approach to Automated Feature Discovery
AI总结：
FAMOSE通过ReAct方法实现自动化特征发现，显著提升分类和回归任务的性能，展示了AI代理在创新解决方案中的有效性。
摘要：
特征工程仍然是机器学习中的一个关键但具有挑战性的瓶颈，特别是在表格数据中，因为从指数级大的特征空间中识别最优特征通常需要大量的领域专业知识。为了解决这一挑战，我们引入了FAMOSE（特征增强和最优选择代理），一种新颖的框架，它利用ReAct范式来自主探索、生成和细化特征，同时在代理架构中集成了特征选择和评估工具。据我们所知，FAMOSE是首次将代理ReAct框架应用于自动化特征工程，特别是回归和分类任务。广泛的实验表明，FAMOSE在分类任务中（尤其是实例超过10000的任务中，ROC-AUC平均增加0.23%），并且在回归任务中通过将RMSE平均减少2.0%而达到最先进的水平，同时比其他算法更稳健。我们假设FAMOSE的强性能是因为ReAct允许LLM上下文窗口记录（通过迭代特征发现和评估步骤）哪些特征有效或无效。这类似于少样本提示，并引导LLM发明更好的、更具创新性的特征。我们的工作提供了证据，表明AI代理在解决需要高度创新解决方案的问题（如特征工程）方面非常有效。
英文摘要：
Feature engineering remains a critical yet challenging bottleneck in machine learning, particularly for tabular data, as identifying optimal features from an exponentially large feature space traditionally demands substantial domain expertise. To address this challenge, we introduce FAMOSE (Feature AugMentation and Optimal Selection agEnt), a novel framework that leverages the ReAct paradigm to autonomously explore, generate, and refine features while integrating feature selection and evaluation tools within an agent architecture. To our knowledge, FAMOSE represents the first application of an agentic ReAct framework to automated feature engineering, especially for both regression and classification tasks. Extensive experiments demonstrate that FAMOSE is at or near the state-of-the-art on classification tasks (especially tasks with more than 10K instances, where ROC-AUC increases 0.23% on average), and achieves the state-of-the-art for regression tasks by reducing RMSE by 2.0% on average, while remaining more robust to errors than other algorithms. We hypothesize that FAMOSE's strong performance is because ReAct allows the LLM context window to record (via iterative feature discovery and evaluation steps) what features did or did not work. This is similar to a few-shot prompt and guides the LLM to invent better, more innovative features. Our work offers evidence that AI agents are remarkably effective in solving problems that require highly inventive solutions, such as feature engineering.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
