ProLaViT:在结构化潜在空间中学习渐进式潜在视觉思维
ProLaViT: Learning Progressive Latent Visual Thoughts in Structured Latent Space
AI总结:
针对多模态大语言模型在复杂视觉推理任务中的不足,ProLaViT框架利用内生自蒸馏机制等,设计两种推理范式及损失函数,使其能在潜在空间进行结构化视觉推导,实验证明效果优于基线。
摘要:
多模态大语言模型(MLLMs)虽有进展,但在复杂视觉推理任务中仍有困难。本文引入ProLaViT框架,使其能在连续潜在空间执行结构化视觉推导。它利用内生自蒸馏机制,构建合成管道,设计两种推理范式,并提出距离加权多样性损失,实验表明其在视觉基准测试中表现优于基线。
英文摘要:
Multimodal Large Language Models (MLLMs) have achieved remarkable progress but still struggle with complex visual reasoning tasks requiring multi-step perception and logical deduction. While explicit visual generation incurs prohibitive computational costs, existing latent approaches often rely on external experts or lack rigorous cognitive logic. In this paper, we introduce ProLaViT (Progressive Latent Visual Thought), a framework empowering MLLMs to perform structured visual derivation in the continuous latent space. Unlike works dependent on heterogeneous external models, ProLaViT leverages an endogenous self-distillation mechanism, utilizing the model's own visual encoder to supervise latent thoughts. To facilitate this, we construct a scalable programmatic synthesis pipeline enabling the model to internalize algorithmic precision without inference time tools. We design two reasoning paradigms: (1) Coarse-to-Fine Causal Chain for spatial tasks, guiding attention from global context to local targets. (2) Dialectical Reasoning Chain for logical tasks, incorporating counter-factual thinking for verification. Furthermore, we propose a Distance-Weighted Diversity Loss to impose topology-aware constraints, preventing feature degeneration by enforcing semantic distinctiveness. Extensive experiments demonstrate that ProLaViT outperforms baselines on vision-centric benchmarks, achieving superior accuracy and interpretability with high efficiency.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。