ReGuLaR: 基于渲染思维链的变分潜在推理
ReGuLaR: Variational Latent Reasoning Guided by Rendered Chain-of-Thought
AI总结：
ReGuLaR通过渲染思维链图像并利用视觉-语义表示正则化后验分布，实现高效的潜在推理，优于现有方法并在多模态推理中超越CoT。
摘要：
尽管思维链（CoT）显著提升了大型语言模型（LLMs）的性能，但显式推理链引入了大量计算冗余。最近的潜在推理方法试图通过将推理过程压缩到潜在空间中来缓解这一问题，但往往由于缺乏适当的压缩指导而导致严重的性能下降。在本研究中，我们提出了Rendered CoT-Guided变分潜在推理（ReGuLaR），一种简单但新颖的潜在学习范式，解决了这一问题。本质上，我们将在变分自动编码（VAE）框架内构建潜在推理，从后验分布中采样当前的潜在推理状态，基于先前的状态。具体来说，当学习这种变分潜在推理模型时，我们将显式推理链渲染为图像，从中提取密集的视觉-语义表示以正则化后验分布，从而实现高效的压缩，同时保持最小的信息损失。广泛的实验表明，ReGuLaR在计算效率和推理有效性方面均显著优于现有的潜在推理方法，并且通过多模态推理甚至超过了CoT，为潜在推理提供了一种新的、有洞察力的解决方案。代码：https://github.com/FanmengWang/ReGuLaR。
英文摘要：
While Chain-of-Thought (CoT) significantly enhances the performance of Large Language Models (LLMs), explicit reasoning chains introduce substantial computational redundancy. Recent latent reasoning methods attempt to mitigate this by compressing reasoning processes into latent space, but often suffer from severe performance degradation due to the lack of appropriate compression guidance. In this study, we propose Rendered CoT-Guided variational Latent Reasoning (ReGuLaR), a simple yet novel latent learning paradigm resolving this issue. Fundamentally, we formulate latent reasoning within the Variational Auto-Encoding (VAE) framework, sampling the current latent reasoning state from the posterior distribution conditioned on previous ones. Specifically, when learning this variational latent reasoning model, we render explicit reasoning chains as images, from which we extract dense visual-semantic representations to regularize the posterior distribution, thereby achieving efficient compression with minimal information loss. Extensive experiments demonstrate that ReGuLaR significantly outperforms existing latent reasoning methods across both computational efficiency and reasoning effectiveness, and even surpasses CoT through multi-modal reasoning, providing a new and insightful solution to latent reasoning. Code: https://github.com/FanmengWang/ReGuLaR.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
