冲突感知融合:通过结构化认知先验缓解大语言模型中的逻辑惯性
Conflict-Aware Fusion: Mitigating Logic Inertia in Large Language Models via Structured Cognitive Priors
AI总结:
针对大语言模型在规则系统结构扰动下表现脆弱的问题,提出冲突感知融合训练流程,通过验证-演绎结构先验和符号推理奖励,在多个压力测试中实现鲁棒性饱和。
摘要:
大型语言模型(LLM)在许多推理基准上取得了高准确率,但在基于规则系统的结构扰动下仍然脆弱。我们引入了一个包含四个压力测试的诊断框架——冗余与必要规则删除、矛盾规则注入、逻辑保持重写和多定律堆叠——并用它来揭示逻辑惯性:生成式LLM(Qwen2/3、TinyLlama、GPT-4o、Gemma-3-4B-IT)和仅编码器BERT基线在矛盾前提下沿学习到的演绎轨迹持续推理的倾向。这种崩溃是剧烈的:未经处理的基线在基础任务上的准确率从1.00下降到矛盾注入时的0.00(实例级精确匹配),而GPT-4o仅解决了56.0%的矛盾案例。我们提出冲突感知融合,这是一个四阶段训练流程,将验证-演绎作为学习到的结构先验强制执行:(i)SFT建立验证前缀;(ii)DPO锐化矛盾停止决策边界;(iii)逻辑不变正则化(LIRE)通过对称KL惩罚逻辑等价规则公式之间的差异;(iv)来自验证反馈的强化学习(RLVF)使用符号前向链接引擎作为确定性预言奖励,联合优化不变性和敏感性。该流程在1.5B和8B骨干网络上均使所有四个主要压力测试达到饱和。我们进一步验证了第二阶段扩展,用Lean 4内核替换命题预言机,在分层187个问题的Lean翻译样本中,对105个经典可推导(T)问题达到99.0%的内核一致性(整体71.7%,涵盖两种极性),为形式化验证的RL训练提供了可靠的升级路径。代码和基准:https://github.com/14H034160212/lemo
英文摘要:
Large language models (LLMs) achieve high accuracy on many reasoning benchmarks but remain brittle under structural perturbations of rule-based systems. We introduce a diagnostic framework with four stress tests -- redundant vs. essential rule deletion, contradictory-rule injection, logic-preserving rewrites, and multi-law stacking -- and use it to expose Logic Inertia: the tendency of generative LLMs (Qwen2/3, TinyLlama, GPT-4o, Gemma-3-4B-IT) and the encoder-only BERT baseline to persist along learned deductive trajectories under inconsistent premises. The collapse is sharp: untreated baselines fall from accuracy 1.00 on the base task to 0.00 on contradiction injection (instance-level exact match), and GPT-4o resolves only 56.0% of contradiction cases. We propose Conflict-Aware Fusion, a four-stage training pipeline that enforces verification-before-deduction as a learned structural prior: (i) SFT establishes the verification preamble; (ii) DPO sharpens the halt-on-contradiction decision boundary; (iii) Logical Invariance REgularisation (LIRE) penalises divergence between logically equivalent rule formulations via symmetric KL; (iv) Reinforcement Learning from Verification Feedback (RLVF) uses a symbolic forward-chaining engine as a deterministic oracle reward, jointly optimising invariance and sensitivity. The pipeline saturates all four primary stress tests for both 1.5B and 8B backbones. We further validate a Phase 2 extension that replaces the propositional oracle with a Lean 4 kernel, attaining 99.0% kernel agreement on the 105 classically-derivable (T) questions within a stratified 187-question Lean-translated sample (overall 71.7% across both polarities), providing a sound upgrade path to formally verified RL training. Code and benchmark: https://github.com/14H034160212/lemo
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。