知识图谱结构作为提示：改进小语言模型在基于知识的因果发现中的能力
Knowledge Graph Structure as Prompt: Improving Small Language Models Capabilities for Knowledge-based Causal Discovery
AI总结：
本文提出KG Structure as Prompt方法，将知识图谱结构信息集成到提示学习中，增强小语言模型在基于知识的因果发现任务中的能力，实验证明其在少样本设置下超越多数基线及传统微调方法。
摘要：
因果发现旨在基于观测数据估计变量间的因果结构。大型语言模型（LLMs）通过推理与变量相关的元数据而非其实际数据值，为解决因果发现问题提供了新视角，该方法被称为基于知识的因果发现。本文研究了小语言模型（SLMs，定义为参数少于10亿的LLMs）在基于提示的学习下进行基于知识的因果发现的能力。具体而言，我们提出了KG Structure as Prompt，一种将知识图谱中的结构信息（如公共邻居节点和元路径）集成到基于提示的学习中以增强SLMs能力的新方法。在少样本设置下，对三类生物医学和开放领域数据集的实验结果表明了该方法的有效性，其超越了大多数基线方法，甚至优于在完整数据集上训练的传统微调方法。我们的发现进一步突显了SLMs的强大能力：结合知识图谱和基于提示的学习，SLMs展现出超越具有更多参数的LLMs的潜力。我们的代码和数据集可在GitHub上获取。
英文摘要：
Causal discovery aims to estimate causal structures among variables based on observational data. Large Language Models (LLMs) offer a fresh perspective to tackle the causal discovery problem by reasoning on the metadata associated with variables rather than their actual data values, an approach referred to as knowledge-based causal discovery. In this paper, we investigate the capabilities of Small Language Models (SLMs, defined as LLMs with fewer than 1 billion parameters) with prompt-based learning for knowledge-based causal discovery. Specifically, we present KG Structure as Prompt, a novel approach for integrating structural information from a knowledge graph, such as common neighbor nodes and metapaths, into prompt-based learning to enhance the capabilities of SLMs. Experimental results on three types of biomedical and open-domain datasets under few-shot settings demonstrate the effectiveness of our approach, surpassing most baselines and even conventional fine-tuning approaches trained on full datasets. Our findings further highlight the strong capabilities of SLMs: in combination with knowledge graphs and prompt-based learning, SLMs demonstrate the potential to surpass LLMs with larger number of parameters. Our code and datasets are available on GitHub.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
