HMVLA：超几何多模态融合用于视觉-语言-动作模型
HMVLA: Hyperbolic Multimodal Fusion for Vision-Language-Action Models
AI总结：
HMVLA通过在双曲空间中融合视觉、语言和动作信息，提升多模态语义对齐的效率和准确性。
摘要：
视觉语言动作（VLA）模型最近在连接多模态感知与机器人控制方面展现出巨大潜力。然而，现有方法往往依赖于对预训练视觉语言模型（VLMs）的直接微调，将语义和视觉特征直接输入策略网络，而未能充分解决VLA领域中的独特语义对齐挑战。在本文中，我们提出了HMVLA，一种新的VLA框架，利用视觉和语言中的内在层次结构实现全面的语义对齐。与传统方法在欧几里得空间中进行对齐不同，我们的HMVLA将多模态特征嵌入到双曲空间中，能够更有效地建模图像文本数据中存在的层次关系。此外，我们引入了一种稀疏门控专家混合（MoE）机制，专门用于语义对齐，从而在增强图像与文本之间的多模态理解的同时提高效率。广泛实验表明，HMVLA在准确性和泛化能力方面均优于基线方法。此外，我们通过重建数据集进一步验证了其鲁棒性，以进一步测试跨领域适应性。
英文摘要：
Vision Language Action (VLA) models have recently shown great potential in bridging multimodal perception with robotic control. However, existing methods often rely on direct fine-tuning of pre-trained Vision-Language Models (VLMs), feeding semantic and visual features directly into a policy network without fully addressing the unique semantic alignment challenges in the VLA domain. In this paper, we propose HMVLA, a novel VLA framework that exploits the inherent hierarchical structures in vision and language for comprehensive semantic alignment. Unlike traditional methods that perform alignment in Euclidean space, our HMVLA embeds multimodal features in hyperbolic space, enabling more effective modeling of the hierarchical relationships present in image text data. Furthermore, we introduce a sparsely gated Mixture of Experts (MoE) mechanism tailored for semantic alignment, which enhances multimodal comprehension between images and text while improving efficiency. Extensive experiments demonstrate that HMVLA surpasses baseline methods in both accuracy and generalization. In addition, we validate its robustness by reconstructing datasets to further test cross domain adaptability.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
