CITI:在不牺牲通用性能的前提下增强大型语言模型的工具使用能力
CITI: Enhancing Tool Utilizing Ability in Large Language Models without Sacrificing General Performance
AI总结:
本文提出CITI方法,通过基于梯度的重要性分数分析模型组件,对重要组件应用MOLoRA、微调次要组件并冻结其余参数,在不牺牲通用性能的前提下有效提升了大型语言模型的工具使用能力。
摘要:
工具学习使大型语言模型(LLMs)能够通过调用工具与外部环境交互,从而丰富LLMs的准确性和能力范围。然而,先前的工作主要集中在提高模型的工具使用准确率及泛化到新工具的能力上,过度迫使LLMs适应特定的工具调用模式,却未考虑对模型通用性能的损害。这偏离了集成工具以增强模型的实际应用和初衷。为解决此问题,我们通过检查隐藏表示的变化和模型组件的基于梯度的重要性分数,剖析了能力权衡。基于分析结果,我们提出了一种基于组件重要性的工具使用能力注入方法(CITI)。根据不同组件的基于梯度的重要性分数,它通过对不同组件应用不同的训练策略,来缓解微调过程引起的能力冲突。CITI对重要组件应用混合LoRA(MOLoRA)。同时,它在LLM的主干网络中微调少数被认为不太重要的组件参数,同时保持其他参数冻结。CITI能够在不过度损害其通用性能的情况下,有效增强模型的工具使用能力。实验结果表明,我们的方法在一系列评估指标上取得了出色的性能。
英文摘要:
Tool learning enables the Large Language Models (LLMs) to interact with the external environment by invoking tools, enriching the accuracy and capability scope of LLMs. However, previous works predominantly focus on improving model's tool-utilizing accuracy and the ability to generalize to new, unseen tools, excessively forcing LLMs to adjust specific tool-invoking pattern without considering the harm to model's general performance. This deviates from the actual applications and original intention of integrating tools to enhance model. To tackle this problem, we dissect the capability trade-offs by examining the hidden representation changes and the gradient-based importance score of model's components. Based on the analysis result, we propose a Component Importance-based Tool-utilizing ability Injection method (CITI). According to the gradient-based importance score of different components, it alleviates the capability conflicts caused by fine-tuning process by applying distinct training strategies to different components. CITI applies Mixture-Of-LoRA (MOLoRA) for important components. Meanwhile, it fine-tunes the parameters of few components deemed less important in the backbone of the LLM, while keeping other parameters frozen. CITI can effectively enhance the model's tool-utilizing capability without excessively compromising its general performance. Experimental results demonstrate that our approach achieves outstanding performance across a range of evaluation metrics.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。