思维反转：利用偏好引导的反向推理预热增强大型语言模型
Reversal of Thought: Enhancing Large Language Models with Preference-Guided Reverse Reasoning Warm-up
AI总结：
提出思维反转（RoT）框架，通过偏好引导的反向推理预热，在推理前增强LLM逻辑能力，实验证明其准确性和效率均优于现有方法。
摘要：
大型语言模型（LLMs）在推理任务中表现出色，但在数学和复杂逻辑推理方面存在局限。现有提升LLMs逻辑能力的方法要么涉及可追踪或可验证的逻辑序列，通过构建逻辑结构生成更可靠的响应，但增加了计算成本；要么引入僵化的逻辑模板规则，降低了灵活性。在本文中，我们提出思维反转（RoT），一种即插即用且成本效益高的推理框架，旨在批量推理前的预热阶段增强LLMs的逻辑推理能力。RoT利用偏好引导的反向推理预热策略，通过元认知机制整合逻辑符号进行伪代码规划，并通过成对偏好自评估，仅凭演示生成任务特定提示，与RLHF塑造的LLMs认知偏好保持一致。通过反向推理，我们利用认知偏好管理器评估知识边界，并通过聚合已知任务的解决方案逻辑和未知任务的风格模板，进一步扩展LLMs的推理能力。跨多种任务的实验表明，RoT在推理准确性和效率上均超越了现有基线。
英文摘要：
Large language models (LLMs) have shown remarkable performance in reasoning tasks but face limitations in mathematical and complex logical reasoning. Existing methods to improve LLMs' logical capabilities either involve traceable or verifiable logical sequences that generate more reliable responses by constructing logical structures yet increase computational costs, or introduces rigid logic template rules, reducing flexibility. In this paper, we propose Reversal of Thought (RoT), a plug-and-play and cost-effective reasoning framework designed to enhance the logical reasoning abilities of LLMs during the warm-up phase prior to batch inference. RoT utilizes a Preference-Guided Reverse Reasoning warm-up strategy, which integrates logical symbols for pseudocode planning through meta-cognitive mechanisms and pairwise preference self-evaluation to generate task-specific prompts solely through demonstrations, aligning with LLMs' cognitive preferences shaped by RLHF. Through reverse reasoning, we utilize a Cognitive Preference Manager to assess knowledge boundaries and further expand LLMs' reasoning capabilities by aggregating solution logic for known tasks and stylistic templates for unknown tasks. Experiments across various tasks demonstrate that RoT surpasses existing baselines in both reasoning accuracy and efficiency.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
