用于提升大语言模型问题求解能力的去混淆因果感知参数高效微调
Deconfounded Causality-aware Parameter-Efficient Fine-Tuning for Problem-Solving Improvement of LLMs
AI总结:
该研究针对大语言模型推理能力不足的问题,构建因果解释框架并提出仅需120万参数的去混淆因果适配微调方法,在多个基准上取得优于基线、可比或优于其他微调方法的效果。
摘要:
大语言模型(LLMs)在依据人类指令处理各类任务方面展现出了显著的效率,但研究表明,它们在数学、物理等需要推理的任务上往往表现不佳。这一局限引发了疑问:LLMs 是真正理解了内嵌的知识,还是仅仅学会了复制词元分布,而并未真正理解内容?本文深入探讨这一问题,旨在增强 LLMs 的推理能力。首先,我们通过在注意力和表征层可视化文本生成过程,探究模型是否具备真正的推理能力。接着,我们将 LLMs 的推理过程构建为一个因果框架,为可视化中观察到的问题提供了正式解释。最后,基于该因果框架,我们提出了去混淆因果适配(Deconfounded Causal Adaptation, DCA)——一种新型参数高效微调(parameter-efficient fine-tuning, PEFT)方法,通过促使模型提取通用问题求解技能并将其应用于不同问题,来增强模型的推理能力。实验表明,我们的方法在多个基准上始终优于基线方法,且仅需120万可训练参数,就能取得与其他微调方法相当甚至更优的结果。这证明了我们的方法在提升 LLMs 整体准确率和可靠性方面的有效性与高效性。
英文摘要:
Large Language Models (LLMs) have demonstrated remarkable efficiency in tackling various tasks based on human instructions, but studies reveal that they often struggle with tasks requiring reasoning, such as math or physics. This limitation raises questions about whether LLMs truly comprehend embedded knowledge or merely learn to replicate the token distribution without a true understanding of the content. In this paper, we delve into this problem and aim to enhance the reasoning capabilities of LLMs. First, we investigate if the model has genuine reasoning capabilities by visualizing the text generation process at the attention and representation level. Then, we formulate the reasoning process of LLMs into a causal framework, which provides a formal explanation of the problems observed in the visualization. Finally, building upon this causal framework, we propose Deconfounded Causal Adaptation (DCA), a novel parameter-efficient fine-tuning (PEFT) method to enhance the model's reasoning capabilities by encouraging the model to extract the general problem-solving skills and apply these skills to different questions. Experiments show that our method outperforms the baseline consistently across multiple benchmarks, and with only 1.2M tunable parameters, we achieve better or comparable results to other fine-tuning methods. This demonstrates the effectiveness and efficiency of our method in improving the overall accuracy and reliability of LLMs.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。