arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.28248cs.AI
收藏

基于结构化潜在轨迹的能量推理

Reasoning as Energy Minimization over Structured Latent Trajectories

David K. Johansson

AI总结:

本文提出EBRM方法,通过能量函数优化多步潜在轨迹,解决单次神经解码和链式思维方法的不足,通过双路径解码训练和潜在锚定缓解分布不匹配问题。

摘要:

本文提出EBRM方法,通过能量函数优化多步潜在轨迹,解决单次神经解码和链式思维方法的不足,通过双路径解码训练和潜在锚定缓解分布不匹配问题。

英文摘要:

Single-shot neural decoders commit to answers without iterative refinement, while chain-of-thought methods introduce discrete intermediate steps but lack a scalar measure of reasoning progress. We propose Energy-Based Reasoning via Structured Latent Planning (EBRM), which models reasoning as gradient-based optimization of a multi-step latent trajectory $z_{1:T}$ under a learned energy function $E(h_x, z)$. The energy decomposes into per-step compatibility, transition consistency, and trajectory smoothness terms. Training combines supervised encoder-decoder learning with contrastive energy shaping using hard negatives, while inference performs gradient descent or Langevin dynamics over $z$ and decodes from $z_T$. We identify a critical failure mode: on CNF logic satisfaction, latent planning reduces accuracy from $\approx 95\%$ to $\approx 56\%$. This degradation arises from a distribution mismatch, where the decoder is trained on encoder outputs $h_x$ but evaluated on planner outputs $z_T$ that drift into unseen latent regions. We analyze this behavior through per-step decoding, latent drift tracking, and gradient decomposition. To address it, we propose dual-path decoder training and latent anchoring. We further introduce a six-part ablation protocol covering component contributions, trajectory length, planner dynamics, initialization, decoder training distribution, and anchor weight. Experiments on three synthetic tasks show that energy decreases monotonically and induces structured latent trajectories on graph and logic tasks, while remaining flat on arithmetic ($r = 0.073$), indicating a negative result. Code is available at https://github.com/dkjo8/ebr-via-structured-latent-planning.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑