发表机构
National University of Singapore; Huazhong University of Science and Technology（新加坡国立大学; 华中科技大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文提出一种三阶段框架，通过提示LLM生成规则并组织成决策树，在少样本表格分类中实现高准确率与可解释性，同时降低推理成本和提示开销。
AI 中文摘要
尽管大型语言模型（LLMs）拥有丰富的世界知识和令人印象深刻的泛化能力，但其直接应用于表格数据分类受到高推理成本和有限可解释性的阻碍。相比之下，决策树快速且透明，但在低数据场景下往往表现不佳。在这项工作中，我们提出了一种新颖框架，通过将LLM知识提炼到可解释的决策树中，在少样本学习设置下弥合这些范式。我们不是直接提示LLM生成完整树（这通常不稳定且效率低下），而是开发了一个三阶段范式，提示LLM生成规则并将规则组织成树。在多个真实世界表格数据集上的实验表明，与现有基线相比，我们的方法在显著降低提示开销的同时，实现了更高的准确性和可解释性。
英文摘要
While Large Language Models (LLMs) possess rich world knowledge and impressive generalization capabilities, their direct application to tabular data classification is hindered by high inference costs and limited interpretability. In contrast, decision trees are fast and transparent but often underperform in low-data regimes. In this work, we propose a novel framework that bridges these paradigms by distilling LLM knowledge into interpretable decision trees under a few-shot learning setting. Instead of directly prompting the LLM to generate full trees, which is often unstable and inefficient, we develop a three-stage paradigm that prompts the LLM to generate rules and organize the rules into a tree. Experiments on multiple real-world tabular datasets demonstrate that our method achieves superior accuracy and interpretability with significantly lower prompting overhead compared to existing baselines.
CommentsAccepted to EMNLP 2026 Main as an oral presentation. Code available: https://github.com/yueqiu0/LLMTree
