ALIVE: 通过对抗学习和指导性语言评估唤醒LLM推理
ALIVE: Awakening LLM Reasoning via Adversarial Learning and Instructive Verbal Evaluation
AI总结：
提出ALIVE框架，通过对抗学习与指导性语言反馈替代标量奖励，使模型内化推理逻辑，在数学、代码和逻辑推理任务中提升准确率、跨域泛化与自我修正能力。
摘要：
大型语言模型（LLM）追求专家级推理的努力一直受到持续的 extit{奖励瓶颈}的阻碍：传统的强化学习（RL）依赖于标量奖励，这些奖励 extbf{成本高昂}难以扩展、 extbf{脆弱}跨领域，并且对解决方案的底层逻辑 extbf{视而不见}。这种对外部、贫乏信号的依赖阻止了模型发展对推理原则的深层、自包含理解。我们引入 extbf{ALIVE}（\emph{对抗学习与指导性语言评估}），一种免人工干预的对齐框架，超越了标量奖励优化，转向内在推理习得。基于\emph{认知协同}原则，ALIVE将问题提出、解决和判断统一在单个策略模型中，以内化正确性的逻辑。通过将对抗学习与指导性语言反馈相结合，ALIVE使模型能够直接从原始语料库内化评估标准，有效将外部批评转化为内生推理能力。在数学推理、代码生成和一般逻辑推理基准上的实证评估表明，ALIVE持续缓解了奖励信号的局限性。在相同数据和计算量下，它实现了准确率提升、显著改善的跨域泛化以及更高的自我修正率。这些结果表明，推理三位一体促进了能力增长的自我维持轨迹，将ALIVE定位为无需人工循环监督的通用推理对齐的可扩展基础。
英文摘要：
The quest for expert-level reasoning in Large Language Models (LLMs) has been hampered by a persistent \textit{reward bottleneck}: traditional reinforcement learning (RL) relies on scalar rewards that are \textbf{costly} to scale, \textbf{brittle} across domains, and \textbf{blind} to the underlying logic of a solution. This reliance on external, impoverished signals prevents models from developing a deep, self-contained understanding of reasoning principles. We introduce \textbf{ALIVE} (\emph{Adversarial Learning with Instructive Verbal Evaluation}), a hands-free alignment framework that moves beyond scalar reward optimization toward intrinsic reasoning acquisition. Grounded in the principle of \emph{Cognitive Synergy}, ALIVE unifies problem posing, solving, and judging within a single policy model to internalize the logic of correctness. By coupling adversarial learning with instructive verbal feedback, ALIVE enables models to internalize evaluative criteria directly from raw corpora, effectively transforming external critiques into an endogenous reasoning faculty. Empirical evaluations across mathematical reasoning, code generation, and general logical inference benchmarks demonstrate that ALIVE consistently mitigates reward signal limitations. With identical data and compute, it achieves accuracy gains, markedly improved cross-domain generalization, and higher self-correction rates. These results indicate that the reasoning trinity fosters a self-sustaining trajectory of capability growth, positioning ALIVE as a scalable foundation for general-purpose reasoning alignment without human-in-the-loop supervision.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
