边缘视觉-语言模型的极端模型压缩：稀疏时间标记融合与自适应神经压缩
Extreme Model Compression for Edge Vision-Language Models: Sparse Temporal Token Fusion and Adaptive Neural Compression
AI总结：
本文提出稀疏时间标记融合和自适应神经压缩技术，实现边缘视觉-语言模型的高效压缩，提升准确率并降低延迟。
摘要：
在视觉-语言任务中对边缘AI的需求要求模型能够在资源受限的设备上实现实时性能，这些设备具有有限的电力和内存。本文提出了两种自适应压缩技术——稀疏时间标记融合（STTF）和自适应神经压缩（ANC）——将算法创新与硬件感知优化相结合。与以往依赖静态剪枝或均匀缩放的方法不同，STTF通过事件驱动的变化检测动态重用视觉标记，而ANC通过学习的路由器条件性地激活编码器分支，从而实现对场景复杂度的细粒度适应。我们的3B参数TinyGPT-STTF在COCO 2017测试集上实现了CIDEr 131.2，BLEU-4 0.38，METEOR 0.31，ROUGE-L 0.56，超过了LLaVA-1.5 7B 17.6个CIDEr点，同时使用2.3倍少的参数和62倍少的设备FLOPs。TinyGPT-ANC实现了CIDEr 128.5。在基于事件的视觉任务中，STTF将平均标记数减少84%（从196到31个），在DVS128手势数据集上保持95.6%的准确性，而ANC在低运动场景中将FLOPs减少高达90%。与强大的基线相比，我们的模型在准确性上提高了高达4.4%，并在延迟上减少了高达13倍。这些结果使有能力的视觉-语言模型能够高效地部署在现实世界的边缘设备上。
英文摘要：
The demand for edge AI in vision-language tasks requires models that achieve real-time performance on resource-constrained devices with limited power and memory. This paper proposes two adaptive compression techniques -- Sparse Temporal Token Fusion (STTF) and Adaptive Neural Compression (ANC) -- that integrate algorithmic innovations with hardware-aware optimizations. Unlike previous approaches relying on static pruning or uniform scaling, STTF dynamically reuses visual tokens through event-driven change detection, while ANC conditionally activates encoder branches via a learned router, enabling fine-grained adaptation to scene complexity. Our 3B-parameter TinyGPT-STTF achieves CIDEr 131.2, BLEU-4 0.38, METEOR 0.31, and ROUGE-L 0.56 on the COCO 2017 test set, surpassing LLaVA-1.5 7B by 17.6 CIDEr points while using 2.3x fewer parameters and 62x fewer on-device FLOPs. TinyGPT-ANC reaches CIDEr 128.5. On event-based vision tasks, STTF reduces average token count by 84% (from 196 to 31 tokens) while preserving 95.6% accuracy on the DVS128 Gesture dataset, and ANC cuts FLOPs by up to 90% in low-motion scenes. Compared to strong baselines, our models improve accuracy by up to 4.4% and reduce latency by up to 13x. These results enable efficient deployment of capable vision-language models on real-world edge devices.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
