PEPS：用于大语言模型中连贯推理轨迹的量子启发强化学习
PEPS: Quantum-Inspired Reinforcement Learning for Coherent Reasoning Traces in LLMs
AI总结：
本研究提出一种基于PEPS保真度奖励的量子启发强化学习方法，通过结构一致性引导训练，显著提升大语言模型在多步推理任务中的连贯性。
摘要：
大语言模型（LLMs）在维持连贯的多步推理轨迹方面常常面临困难，尤其是在需要结构化逻辑流程的任务中。本研究引入了一种量子启发的方法来应对这一挑战，通过将源自投影纠缠对态（PEPS）的保真度奖励纳入近端策略优化中。与先前使用直接监督或对比目标的方法不同，所提出的方法通过结构一致性来引导学习，提供了一种在生成的推理轨迹中强制实现全局连贯性的新颖途径。所提出的框架在多个连贯性判定指标上进行了评估，使用了涵盖算术、直觉和基于蕴含推理的多样化数据集，如GSM8K、StrategyQA和EntailmentBank。结果表明，所提出的量子启发方法在监督、对比和预训练基线方法之上提供了显著改进，突显了量子启发保真度作为提升大语言模型中推理轨迹连贯性基础的有效性。
英文摘要：
Large Language Models (LLMs) often struggle with maintaining coherent multi-step reasoning traces, particularly in tasks that require a structured logical flow. This work introduces a quantum-inspired approach to address the challenge by incorporating a fidelity-based reward derived from Projected Entangled Pair States (PEPS) into Proximal Policy Optimization. Unlike prior approaches that use direct supervision or contrastive objectives, the proposed method guides learning through structural consistency, offering a novel approach to enforce global coherence in generated reasoning traces. The proposed framework is evaluated using multiple coherence-determining metrics on diverse datasets such as GSM8K, StrategyQA, and EntailmentBank spanning arithmetic, intuitive, and entailment-based reasoning. Results show that the proposed quantum-inspired approach offers significant improvements over supervised, contrastive, and pretrained baseline approaches, highlighting the effectiveness of quantum-inspired fidelity as a foundation to improve reasoning trace coherence in LLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
