SIGMA：基于SHAP的无元数据LLM自动特征工程隐式轨迹生成
SIGMA: SHAP-Guided Implicit-Trajectory Generation for Metadata-Free LLM-Based AutoFE
查看机构详情
- Yokohama National University（横滨国立大学）
机构由 AI 辅助整理，请以论文原文为准。
浏览论文内容
中文总结 AI 辅助
针对无元数据场景下LLM-AutoFE的轨迹问题，提出SIGMA框架，利用SHAP值和EXIT方法，降低特征重复率并提升效率，实现与SOTA相当的性能。
中文摘要 AI 辅助
近期研究利用大型语言模型（LLM）通过语义描述和基于轨迹的提示来增强自动特征工程（AutoFE），但存在两个限制其在长序列优化中适用性与可扩展性的挑战：其一，许多实际场景中缺乏语义元数据；其二，轨迹积累会增加超出上下文窗口的风险，而若无轨迹则生成过程可能不稳定，导致陷入局部最优且生成特征的重复率较高。为此，我们提出SIGMA（基于SHAP的无元数据AutoFE隐式轨迹生成），这是一个可扩展的恒定上下文优化框架。SIGMA利用SHAP值提供任务感知信号以指导分组特征生成，替代语义信息；此外，我们采用EXposed-feature隐式轨迹（EXIT）方法，其中提示中的暴露特征隐式表示轨迹。实验结果表明，SIGMA在提示长度几乎恒定的情况下，实现了与最先进（SOTA）LLM基线相当的性能；值得注意的是，EXIT将生成特征的重复率从37.2%降至6.8%，同时SIGMA仅用平均5.4个特征就达到了传统SOTA性能，在特征利用方面展现出显著的效率提升。
英文摘要
Recent research has leveraged Large Language Models (LLMs) to enhance Automated Feature Engineering (AutoFE) through semantic descriptions and trajectory-based prompting. However, there exist two challenges that limit their applicability and scalability in long-horizon optimization: (1) semantic metadata is unavailable in many practical settings, and (2) trajectory accumulation increases the risk of exceeding the context window, while without it, the generation process can become unstable, leading to becoming stuck in the local optima and a high duplicate rate of generated features. To this end, we propose a SHAP-enhanced Implicit-trajectory Generation for Metadata-free AutoFE (SIGMA), a scalable constant-context optimization framework. SIGMA leverages SHAP values to provide task-aware signals for guiding group feature generation instead of semantic information. In addition, we adopt an EXposed-feature Implicit Trajectory (EXIT) approach, where the exposed features in the prompt implicitly represent the trajectory. Empirical results demonstrate that SIGMA achieves performance comparable to the state-of-the-art (SOTA) LLM baselines with a nearly constant prompt length. Notably, EXIT significantly reduces the duplicate ratio of generated features from 37.2% to 6.8%. At the same time, SIGMA matches traditional SOTA performance with only 5.4 features on average, demonstrating substantial efficiency gains in feature utilization.
