MagicVL-2B：通过课程学习利用轻量级视觉编码器赋能移动设备上的视觉语言模型
MagicVL-2B: Empowering Vision-Language Models on Mobile Devices with Lightweight Visual Encoders via Curriculum Learning
AI总结：
MagicVL-2B通过轻量级视觉编码器和多模态课程学习策略，在保持准确率的同时降低设备端功耗41.1%，实现移动端高效视觉语言推理。
摘要：
视觉语言模型（VLMs）近年来取得了显著突破，实现了日常生活中的多样化应用。然而，视觉语言模型巨大的计算和存储需求对其在移动设备上的高效部署构成了重大挑战，而移动设备是当今最普遍且最易获取的计算平台。在这项工作中，我们介绍了MagicVL-2B，一种为旗舰智能手机精心优化的新型视觉语言模型。MagicVL-2B利用参数少于1亿的轻量级视觉编码器，并采用重新设计的动态分辨率方案，该方案能够自适应地生成图像令牌，而无需过度修改图像尺寸。为了进一步增强这种紧凑编码器在视觉语言模型中的性能，我们提出了一种多模态课程学习策略，该策略在整个训练过程中逐步增加任务难度和数据信息密度。这种方法显著提升了模型在各种子任务上的性能。在标准视觉语言模型基准上的广泛评估表明，MagicVL-2B在匹配当前最先进模型准确率的同时，将设备端功耗降低了41.1%。这些结果确立了MagicVL-2B作为现实世界移动视觉语言应用的实用且稳健的解决方案，使先进的多模态智能能够直接在智能手机上运行。
英文摘要：
Vision-Language Models (VLMs) have achieved remarkable breakthroughs in recent years, enabling a diverse array of applications in everyday life. However, the substantial computational and storage demands of VLMs pose significant challenges for their efficient deployment on mobile devices, which represent the most ubiquitous and accessible computing platforms today. In this work, we introduce MagicVL-2B, a novel VLM meticulously optimized for flagship smartphones. MagicVL-2B leverages a lightweight visual encoder with fewer than 100M parameters and features a redesigned dynamic resolution scheme that adaptively generates image tokens without excessive modification of image dimensions. To further enhance the performance of this compact encoder within VLMs, we propose a multimodal curriculum learning strategy that incrementally increases task difficulty and data information density throughout training. This approach substantially improves the model's performance across a variety of sub-tasks. Extensive evaluations on standard VLM benchmarks demonstrate that MagicVL-2B matches the accuracy of current state-of-the-art models while reducing on-device power consumption by 41.1%. These results establish MagicVL-2B as a practical and robust solution for real-world mobile vision-language applications, enabling advanced multimodal intelligence to run directly on smartphones.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
