CogniVerse: 用认知反思与几何推理革新多模态检索增强生成
CogniVerse: Revolutionizing Multi-Modal Retrieval-Augmented Generation with Cognitive Reflection and Geometric Reasoning
AI总结：
提出CogniVerse框架，通过认知反思模块、基于黎曼流形对齐的多模态检索模块和最优传输层次生成模块，解决多模态检索增强生成中的噪声检索、跨模态语义错位和生成不连贯问题。
摘要：
多模态检索增强生成（MMRAG）已成为一种强大的范式，通过整合外部视觉、文本和结构知识，增强多模态大语言模型在知识密集型问答中的能力。然而，现有的MMRAG框架存在关键限制，包括噪声和无关检索、跨模态语义错位、缺乏自适应推理以及局部和全局上下文生成不连贯。我们提出了 extbf{CogniVerse}，一种新颖的MMRAG框架，通过受认知启发的数学严谨方法解决这些挑战。借鉴人类推理，CogniVerse集成了三个协同组件：（1）认知反思模块，动态评估检索必要性并过滤相关多模态内容，减少噪声和计算开销；（2）多模态检索模块，使用信息几何在黎曼流形中对齐嵌入，并通过谱图理论优化知识图谱，确保精确且连贯的检索；（3）层次生成模块，采用基于最优传输的损失来平衡词元级准确性和全局语义连贯性。大量实验表明，CogniVerse在准确性和连贯性上显著优于最先进系统，同时降低了检索延迟。
英文摘要：
Multi-modal Retrieval-Augmented Generation (MMRAG) has emerged as a powerful paradigm for enhancing Multimodal Large Language Models in knowledge-intensive question answering by integrating external visual, textual, and structural knowledge. However, existing MMRAG frameworks suffer from critical limitations, including noisy and irrelevant retrieval, cross-modal semantic misalignment, lack of adaptive reasoning, and incoherent generation across local and global contexts. We introduce \textbf{CogniVerse}, a novel MMRAG framework that addresses these challenges through a cognitive-inspired, mathematically rigorous approach. Drawing from human-like reasoning, CogniVerse integrates three synergistic components: (1) a Cognitive Reflection Module that dynamically assesses retrieval necessity and filters relevant multi-modal content, reducing noise and computational overhead; (2) a Multi-modal Retrieval Module that aligns embeddings in a Riemannian manifold using information geometry and refines knowledge graphs via spectral graph theory, ensuring precise and coherent retrieval; and (3) a Hierarchical Generation Module that employs an optimal transport-based loss to balance token-level accuracy and global semantic coherence. Extensive experiments demonstrate that CogniVerse significantly outperforms state-of-the-art systems in both accuracy and coherence, while reducing retrieval latency.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
