思维中的推理：潜在空间中的动态多模态交错
Reasoning Within the Mind: Dynamic Multimodal Interleaving in Latent Space
AI总结：
本文提出DMLR框架，通过动态潜在策略梯度优化和视觉注入策略，在潜在空间中实现视觉与文本的动态交错推理，提升多模态推理性能并保持高效推理。
摘要：
近期多模态大语言模型（MLLMs）的进步显著增强了跨模态理解和推理，通过在语义空间中整合链式思维（CoT）推理。在此基础上，近期研究将CoT机制扩展到视觉模态，使模型能够通过外部工具或显式图像生成整合视觉信息。然而，这些方法仍依赖显式分步推理，存在感知-推理交互不稳定和显著的计算开销。受人类认知启发，我们提出思维并非线性展开，而是在潜在空间中通过动态推理与感知的交错进行。受此观点启发，我们提出DMLR，一种测试时动态多模态潜在推理框架，采用基于置信度的潜在策略梯度优化来优化潜在思考令牌以深入推理。此外，引入了动态视觉注入策略，该策略在每个潜在思考令牌中检索最相关的视觉特征，并更新最佳视觉片段集。更新的片段随后注入到潜在思考令牌中，以实现动态视觉-文本交错。在七个多模态推理基准和各种模型架构上的实验表明，DMLR显著提高了推理和感知性能，同时保持高效的推理效率。
英文摘要：
Recent advancements in Multimodal Large Language Models (MLLMs) have significantly enhanced cross-modal understanding and reasoning by incorporating Chain-of-Thought (CoT) reasoning in the semantic space. Building upon this, recent studies extend the CoT mechanism to the visual modality, enabling models to integrate visual information during reasoning through external tools or explicit image generation. However, these methods remain dependent on explicit step-by-step reasoning, unstable perception-reasoning interaction and notable computational overhead. Inspired by human cognition, we posit that thinking unfolds not linearly but through the dynamic interleaving of reasoning and perception within the mind. Motivated by this perspective, we propose DMLR, a test-time Dynamic Multimodal Latent Reasoning framework that employs confidence-guided latent policy gradient optimization to refine latent think tokens for in-depth reasoning. Furthermore, a Dynamic Visual Injection Strategy is introduced, which retrieves the most relevant visual features at each latent think token and updates the set of best visual patches. The updated patches are then injected into latent think token to achieve dynamic visual-textual interleaving. Experiments across seven multimodal reasoning benchmarks and various model architectures demonstrate that DMLR significantly improves reasoning and perception performance while maintaining high inference efficiency.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
