LTA-thinker: 用于复杂推理的大语言模型的潜在思维增强训练框架
LTA-thinker: Latent Thought-Augmented Training Framework for Large Language Models on Complex Reasoning
AI总结：
LTA-Thinker通过提升潜在思维分布方差和信息效率，优化大语言模型的复杂推理性能。
摘要：
大语言模型中的复杂推理可以通过测试时间缩放（TTS）动态优化，以缓解过度思考问题。诸如Coconut、SoftCoT及其变体等方法在连续潜在空间推理中效果显著，但核心瓶颈仍在于高质量潜在思维的高效生成和利用。借鉴SoftCoT++理论，认为生成的潜在思维分布方差越大，越能逼近黄金真理分布，我们提出了一种潜在思维增强训练框架——LTA-Thinker，从两个角度提升分布方差并提高推理性能。首先，LTA-Thinker基于可学习先验构建了潜在思维生成架构，旨在增加生成的潜在思维向量的分布方差，以简化整体结构并提升性能上限。其次，LTA-Thinker引入了基于分布的方向优化范式，通过联合约束分布局部性和分布尺度，改进信息效率和计算成本，通过多目标协同训练策略，结合标准监督微调（SFT）损失与两种新型损失：语义对齐损失，利用KL散度确保潜在思维与问题语义高度相关；推理聚焦损失，利用对比学习机制引导模型聚焦最关键推理步骤。实验显示，LTA-thinker在各种基线中实现了最先进的（SOTA）性能，展示了更高的性能上限和更好的扩展效果。
英文摘要：
Complex Reasoning in Large Language Models can be dynamically optimized using Test-Time Scaling (TTS) to mitigate Overthinking. Methods such as Coconut, SoftCoT and its variant are effective in continuous latent space inference, the core bottleneck still lies in the efficient generation and utilization of high-quality Latent Thought. Drawing from the theory of SoftCoT++ that a larger variance in the generated Latent Thought distribution more closely approximates the golden truth distribution, we propose a Latent Thought-Augmented Training Framework--LTA-Thinker, which improves distributional variance and enhances reasoning performance from two perspectives. First, LTA-Thinker constructs a Latent Thought generation architecture based on a learnable prior. This architecture aims to increase the variance distribution of generated Latent Thought Vectors in order to simplify the overall structure and raise the performance ceiling. Second, LTA-Thinker introduces a distribution-based directional optimization paradigm that jointly constrains both distribution locality and distribution scale. This mechanism improves information efficiency and computational cost through a multi-objective co-training strategy, which combines standard Supervised Fine-Tuning (SFT) loss with two novel losses: Semantic Alignment Loss, which utilizes KL divergence to ensure that the Latent Thought is highly relevant to the semantics of the question; Reasoning Focus Loss, which utilizes a contrastive learning mechanism to guide the model to focus on the most critical reasoning steps. Experiments show that LTA-thinker achieves state-of-the-art (SOTA) performance among various baselines and demonstrates a higher performance ceiling and better scaling effects.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
