使用思维链提示大型语言模型进行少样本知识库问题生成
Prompting Large Language Models with Chain-of-Thought for Few-Shot Knowledge Base Question Generation
AI总结：
本文提出基于思维链的提示方法 KQG-CoT 及其改进版 KQG-CoT+，将知识库问题生成转化为推理问题，通过检索逻辑形式并按复杂度排序构建推理链，在少样本场景下显著优于基线并在 PathQuestions 数据集上大幅超越现有 SoTA。
摘要：
知识库问题生成（KBQG）任务旨在将逻辑形式转换为自然语言问题。由于大规模问题标注成本高昂，亟需开发低资源场景下的 KBQG 方法。然而，现有方法严重依赖标注数据进行微调，不适用于少样本问题生成。大型语言模型（LLMs）的出现展现了其在少样本任务中令人瞩目的泛化能力。受作为推理上下文学习策略的思维链（CoT）提示启发，我们将 KBQG 任务转化为一个推理问题，其中完整问题的生成被拆分为一系列子问题生成。我们提出的提示方法 KQG-CoT 首先考虑逻辑形式的特征，从无标签数据池中检索支持性逻辑形式。然后，我们编写提示，基于所选示例明确生成复杂问题的推理链。为进一步确保提示质量，我们通过按逻辑形式的复杂度对其进行排序，将 KQG-CoT 扩展为 KQG-CoT+。我们在三个公开的 KBQG 数据集上进行了广泛实验。结果表明，我们的提示方法在评估数据集上持续优于其他提示基线。值得注意的是，我们的 KQG-CoT+ 方法在 PathQuestions 数据集上分别以 18.25、10.72 和 10.18 个绝对百分点超越了现有的少样本 SoTA 结果，涵盖 BLEU-4、METEOR 和 ROUGE-L 指标。
英文摘要：
The task of Question Generation over Knowledge Bases (KBQG) aims to convert a logical form into a natural language question. For the sake of expensive cost of large-scale question annotation, the methods of KBQG under low-resource scenarios urgently need to be developed. However, current methods heavily rely on annotated data for fine-tuning, which is not well-suited for few-shot question generation. The emergence of Large Language Models (LLMs) has shown their impressive generalization ability in few-shot tasks. Inspired by Chain-of-Thought (CoT) prompting, which is an in-context learning strategy for reasoning, we formulate KBQG task as a reasoning problem, where the generation of a complete question is splitted into a series of sub-question generation. Our proposed prompting method KQG-CoT first retrieves supportive logical forms from the unlabeled data pool taking account of the characteristics of the logical form. Then, we write a prompt to explicit the reasoning chain of generating complicated questions based on the selected demonstrations. To further ensure prompt quality, we extend KQG-CoT into KQG-CoT+ via sorting the logical forms by their complexity. We conduct extensive experiments over three public KBQG datasets. The results demonstrate that our prompting method consistently outperforms other prompting baselines on the evaluated datasets. Remarkably, our KQG-CoT+ method could surpass existing few-shot SoTA results of the PathQuestions dataset by 18.25, 10.72, and 10.18 absolute points on BLEU-4, METEOR, and ROUGE-L, respectively.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
