arXivDaily arXiv每日学术速递 周一至周五更新
北美退伍军人节(Veterans Day),12 日无更新
arXiv 2602.20980cs.CVcs.AI
收藏

CrystaL: MLLMs中视觉潜在特征的自发涌现

CrystaL: Spontaneous Emergence of Visual Latents in MLLMs

Yang Zhang, Danyang Li, Yuxuan Li, Xin Zhang, Tianyu Xie, Mingming Cheng, Xiang Li

AI总结:

CrystaL通过显式对齐注意力模式和预测分布,实现视觉潜在特征的自发涌现,提升细粒度视觉理解和推理能力。

摘要:

多模态大语言模型(MLLMs)通过整合强大的语言骨干与大规模视觉编码器实现了显著的性能提升。其中,潜在链式思维(CoT)方法通过在连续隐藏状态中实现隐式推理,促进了无缝的视觉-语言整合和更快的推理。然而,现有潜在Cot中启发式预定义的监督信号在保持中间潜在状态中的关键视觉信息方面提供了有限的指导。为了解决这一限制,我们提出了CrystaL(结晶化潜在推理),一个单阶段框架,分别处理完整和损坏的图像。通过显式对齐两个路径的注意力模式和预测分布,CrystaL将潜在表示结晶化为任务相关的视觉语义,而无需依赖辅助注释或外部模块。在感知密集基准上的广泛实验表明,CrystaL在细粒度视觉理解方面一致优于最先进的基线,同时保持了稳健的推理能力。

英文摘要:

Multimodal Large Language Models (MLLMs) have achieved remarkable performance by integrating powerful language backbones with large-scale visual encoders. Among these, latent Chain-of-Thought (CoT) methods enable implicit reasoning in continuous hidden states, facilitating seamless vision-language integration and faster inference. However, existing heuristically predefined supervision signals in latent CoT provide limited guidance for preserving critical visual information in intermediate latent states. To address this limitation, we propose CrystaL (Crystallized Latent Reasoning), a single-stage framework with two paths to process intact and corrupted images, respectively. By explicitly aligning the attention patterns and prediction distributions across the two paths, CrystaL crystallizes latent representations into task-relevant visual semantics, without relying on auxiliary annotations or external modules. Extensive experiments on perception-intensive benchmarks demonstrate that CrystaL consistently outperforms state-of-the-art baselines, achieving substantial gains in fine-grained visual understanding while maintaining robust reasoning capabilities.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑