arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.12013cs.AI
收藏

InjectRBP: 通过模式注入引导大语言模型推理行为

InjectRBP: Steering Large Language Model Reasoning Behavior via Pattern Injection

Xi'an Jiaotong University(西安交通大学);University of Southampton(南安普顿大学);University of Hongkong(香港大学)

Xiuping Wu, Zhao Yu, Yuxin Cheng, Ngai Wong, Liangjun Ke, Tapas Mishra, Konstantinos V. Katsikopoulos

AI总结:

InjectRBP通过注入行为模式提升大语言模型推理能力,无需参数更新,实验显示在多种推理任务中性能提升显著。

摘要:

推理可以显著提升大语言模型的性能。尽管最近的研究已经利用与行为相关的提示调整来增强推理,但这些设计仍然主要依赖直观方法,缺乏对底层行为模式的系统分析。受此启发,我们从行为模式的角度研究模型推理行为如何塑造推理过程。我们观察到,当模型响应特定类型的问题时,其推理行为会表现出适应性的分布,并且在结构上注入这些模式可以显著影响模型推理过程和结果的质量。基于这些发现,我们提出了两种不需要参数更新的优化方法:InjectCorrect和InjectRLOpt。InjectCorrect通过模仿自身过去正确答案中推导出的行为模式来引导模型。InjectRLOpt从历史行为模式数据中学习价值函数,并通过我们提出的方法Reliability-Aware Softmax Policy,在推理过程中生成行为注入物以引导推理过程。我们的实验表明,这两种方法可以在各种推理任务中提升模型性能,而无需对模型参数进行任何修改,分别实现了高达5.34%和8.67%的提升。

英文摘要:

Reasoning can significantly enhance the performance of Large Language Models. While recent studies have exploited behavior-related prompts adjustment to enhance reasoning, these designs remain largely intuitive and lack a systematic analysis of the underlying behavioral patterns. Motivated by this, we investigate how models' reasoning behaviors shape reasoning from the perspective of behavioral patterns. We observe that models exhibit adaptive distributions of reasoning behaviors when responding to specific types of questions, and that structurally injecting these patterns can substantially influence the quality of the models' reasoning processes and outcomes. Building on these findings, we propose two optimization methods that require no parameter updates: InjectCorrect and InjectRLOpt. InjectCorrect guides the model by imitating behavioral patterns derived from its own past correct answers. InjectRLOpt learns a value function from historical behavior-pattern data and, via our proposed Reliability-Aware Softmax Policy, generates behavioral injectant during inference to steer the reasoning process. Our experiments demonstrate that both methods can improve model performance across various reasoning tasks without requiring any modifications to model parameters, achieving gains of up to 5.34% and 8.67%, respectively.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑