arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2501.16247cs.LGcs.CL
收藏

基于大语言模型的零样本决策树构建

Zero-Shot Decision Tree Construction via Large Language Models

University of Chile(智利大学);National Center for Artificial Intelligence (CENIA)(国家人工智能中心(CENIA))

Lucas Carrasco, Felipe Urrutia, Andrés Abeliuk

AI总结:

本文提出一种基于CART原则利用大语言模型预训练知识构建零样本决策树的新算法,无需训练数据即可执行属性离散化和Gini指数计算,在表格数据集上性能优于基线零样本方法且媲美有监督方法,为低数据机器学习提供了可解释的知识驱动新基线。

摘要:

本文提出了一种基于分类与回归树(CART)原则,以零样本方式使用大语言模型(LLMs)构建决策树的新算法。传统决策树归纳方法严重依赖带标签数据,利用信息增益或Gini指数等标准递归地划分数据。相比之下,我们提出了一种利用LLMs中嵌入的预训练知识来构建决策树的方法,无需训练数据。我们的方法利用LLMs执行决策树构建所必需的操作,包括属性离散化、概率计算以及基于这些概率的Gini指数计算。我们表明,这些零样本决策树能够超越基线零样本方法,并在表格数据集上取得与有监督数据驱动决策树相媲美的性能。通过该方法构建的决策树提供了透明且可解释的模型,在解决数据稀缺问题的同时保持了可解释性。这项工作在低数据机器学习领域确立了新的基线,为数据驱动的树构建提供了一种有原则的、知识驱动的替代方案。

英文摘要:

This paper introduces a novel algorithm for constructing decision trees using large language models (LLMs) in a zero-shot manner based on Classification and Regression Trees (CART) principles. Traditional decision tree induction methods rely heavily on labeled data to recursively partition data using criteria such as information gain or the Gini index. In contrast, we propose a method that uses the pre-trained knowledge embedded in LLMs to build decision trees without requiring training data. Our approach leverages LLMs to perform operations essential for decision tree construction, including attribute discretization, probability calculation, and Gini index computation based on the probabilities. We show that these zero-shot decision trees can outperform baseline zero-shot methods and achieve competitive performance compared to supervised data-driven decision trees on tabular datasets. The decision trees constructed via this method provide transparent and interpretable models, addressing data scarcity while preserving interpretability. This work establishes a new baseline in low-data machine learning, offering a principled, knowledge-driven alternative to data-driven tree construction.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑