频谱上的推理：将大语言模型对齐到系统1与系统2思维
Reasoning on a Spectrum: Aligning LLMs to System 1 and System 2 Thinking
AI总结：
本研究通过构建含系统1和系统2答案的数据集对齐LLM，发现系统2擅长算术与符号推理，系统1擅长常识推理，并基于生成熵动态组合两者，无需额外训练即可在多数基准上超越单一模型。
摘要：
大语言模型（LLMs）展现出令人印象深刻的推理能力，然而它们对结构化逐步处理的依赖揭示了一个关键局限。相比之下，人类认知会根据情境在直觉性、启发式的（系统1）推理与分析性、深思熟虑的（系统2）推理之间灵活切换。人类认知灵活性与LLMs对单一推理风格的依赖之间的这种差异引发了一个关键问题：虽然人类快速启发式推理因其效率和适应性而进化，但统一的推理方式对LLMs而言是否真正最优，还是说这种不灵活性使它们在面对需要更敏捷、直觉性响应的任务时变得脆弱和不可靠？为了回答这些问题，我们通过整理一个包含有效系统1和系统2答案的数据集，明确地将LLMs对齐到这些推理风格，并评估它们在推理基准上的表现。我们的结果揭示了准确性与效率之间的权衡：对齐系统2的模型在算术和符号推理方面表现出色，而对齐系统1的模型在常识推理任务中表现更好。为了分析推理频谱，我们通过改变对齐数据的比例在两个极端之间进行插值，这导致了准确性的单调变化。对模型响应的机制分析表明，系统1模型采用更确定的输出，而系统2模型表现出更大的不确定性。基于这些发现，我们进一步根据其生成结果的熵，无需额外训练，将系统1和系统2对齐的模型组合起来，得到一个在几乎所有基准上表现更优的动态模型。这项工作挑战了逐步推理总是最优的假设，并强调了根据任务需求调整推理策略的必要性。
英文摘要：
Large Language Models (LLMs) exhibit impressive reasoning abilities, yet their reliance on structured step-by-step processing reveals a critical limitation. In contrast, human cognition fluidly adapts between intuitive, heuristic (System 1) and analytical, deliberative (System 2) reasoning depending on the context. This difference between human cognitive flexibility and LLMs' reliance on a single reasoning style raises a critical question: while human fast heuristic reasoning evolved for its efficiency and adaptability, is a uniform reasoning approach truly optimal for LLMs, or does its inflexibility make them brittle and unreliable when faced with tasks demanding more agile, intuitive responses? To answer these questions, we explicitly align LLMs to these reasoning styles by curating a dataset with valid System 1 and System 2 answers, and evaluate their performance across reasoning benchmarks. Our results reveal an accuracy-efficiency trade-off: System 2-aligned models excel in arithmetic and symbolic reasoning, while System 1-aligned models perform better in commonsense reasoning tasks. To analyze the reasoning spectrum, we interpolated between the two extremes by varying the proportion of alignment data, which resulted in a monotonic change in accuracy. A mechanistic analysis of model responses shows that System 1 models employ more definitive outputs, whereas System 2 models demonstrate greater uncertainty. Building on these findings, we further combine System 1- and System 2-aligned models based on the entropy of their generations, without additional training, and obtain a dynamic model that outperforms across nearly all benchmarks. This work challenges the assumption that step-by-step reasoning is always optimal and highlights the need for adapting reasoning strategies based on task demands.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
