arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2401.13849cs.CLcs.AI
收藏

TPD:通过原则发现与引导增强学生语言模型的推理能力

TPD: Enhancing Student Language Model Reasoning via Principle Discovery and Guidance

Georgia Institute of Technology(佐治亚理工学院);University of California, Los Angeles(加州大学洛杉矶分校)

Haorui Wang, Rongzhi Zhang, Yinghao Li, Lingkai Kong, Yuchen Zhuang, Xiusi Chen, Chao Zhang

AI总结:

本文提出 TPD 师生框架,由教师模型依据学生错误提炼原则并选择示例,使其在推理阶段无需外部干预即可学习,八项任务平均较思维链提示提升 6.2%。

摘要:

大语言模型(LLM)最近展现出卓越的推理能力。然而,较大的模型在推理任务中往往优于较小的同类模型,这带来了如何有效迁移这些能力的挑战。现有方法严重依赖大量微调数据,或在推理期间持续与更优秀的教师 LLM 交互。我们提出一个名为“通过原则发现进行教学”(Teaching via Principle Discovery,TPD)的基于原则的师生框架,以解决这些局限。受人类学习机制启发,TPD 使用基于原则的方法模拟教师与学生之间的互动。教师 LLM 根据学生 LLM 的错误生成解题指导和纠错原则。这些原则指导对指导内容的精炼,以及从验证集中选择有启发性的示例。这使学生模型能够同时从教师的引导和自身错误中学习。一旦学生模型开始进行推理,TPD 不再需要教师 LLM 或人类的进一步干预。通过在八个推理任务上的大量实验,我们证明了 TPD 的有效性。与标准思维链提示相比,TPD 显著提升了学生模型的性能,平均实现 6.2% 的提升。

英文摘要:

Large Language Models (LLMs) have recently showcased remarkable reasoning abilities. However, larger models often surpass their smaller counterparts in reasoning tasks, posing the challenge of effectively transferring these capabilities from larger models. Existing approaches heavily rely on extensive fine-tuning data or continuous interactions with a superior teacher LLM during inference. We introduce a principle-based teacher-student framework called ``Teaching via Principle Discovery'' (TPD) to address these limitations. Inspired by human learning mechanisms, TPD mimics the interaction between a teacher and a student using a principle-based approach. The teacher LLM generates problem-solving instructions and corrective principles based on the student LLM's errors. These principles guide the refinement of instructions and the selection of instructive examples from a validation set. This enables the student model to learn from both the teacher's guidance and its own mistakes. Once the student model begins making inferences, TPD requires no further intervention from the teacher LLM or humans. Through extensive experiments across eight reasoning tasks, we demonstrate the effectiveness of TPD. Compared to standard chain-of-thought prompting, TPD significantly improves the student model's performance, achieving $6.2\%$ improvement on average.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑