在边缘侧释放MLLMs：通过自适应SVD蒸馏实现跨模态重识别的统一框架
Unleashing MLLMs on the Edge: A Unified Framework for Cross-Modal ReID via Adaptive SVD Distillation
AI总结：
本文提出MLLMEmbed-ReID框架，通过自适应SVD蒸馏实现跨模态重识别的统一部署，结合云-边缘架构提升性能与效率。
摘要：
在实际中，跨模态重识别（CM-ReID）的云-边缘部署面临挑战，因为需要维护一个碎片化的专用云模型生态系统，以适应不同的模态。尽管多模态大语言模型（MLLMs）具有强大的统一潜力，但现有方法未能将它们适配成单一的端到端骨干网络，并且缺乏有效的知识蒸馏策略以适应边缘部署。为了解决这些限制，我们提出了MLLMEmbed-ReID，一个基于强大云-边缘架构的统一框架。首先，我们将基础的MLLM适配成最先进的云模型。我们利用基于指令的提示来指导MLLM生成跨RGB、红外、草图和文本模态的统一嵌入空间。然后，该模型通过分层低秩适应微调（LoRA-SFT）策略高效训练，优化在整体跨模态对齐目标下。第二，为了将知识部署到边缘原生的学生模型上，我们引入了一种受教师特征空间中低秩属性启发的新蒸馏策略。为了优先考虑关键信息，该方法采用主成分映射损失，同时通过特征关系损失保留关系结构。我们的轻量级边缘模型在多个视觉CM-ReID基准上实现了最先进的性能，而其云端对应物在所有CM-ReID基准上均表现出色。因此，MLLMEmbed-ReID框架为在资源受限设备上部署统一的MLLM级智能提供了一个完整而有效的解决方案。代码和模型将很快开源。
英文摘要：
Practical cloud-edge deployment of Cross-Modal Re-identification (CM-ReID) faces challenges due to maintaining a fragmented ecosystem of specialized cloud models for diverse modalities. While Multi-Modal Large Language Models (MLLMs) offer strong unification potential, existing approaches fail to adapt them into a single end-to-end backbone and lack effective knowledge distillation strategies for edge deployment. To address these limitations, we propose MLLMEmbed-ReID, a unified framework based on a powerful cloud-edge architecture. First, we adapt a foundational MLLM into a state-of-the-art cloud model. We leverage instruction-based prompting to guide the MLLM in generating a unified embedding space across RGB, infrared, sketch, and text modalities. This model is then trained efficiently with a hierarchical Low-Rank Adaptation finetuning (LoRA-SFT) strategy, optimized under a holistic cross-modal alignment objective. Second, to deploy its knowledge onto an edge-native student, we introduce a novel distillation strategy motivated by the low-rank property in the teacher's feature space. To prioritize essential information, this method employs a Principal Component Mapping loss, while relational structures are preserved via a Feature Relation loss. Our lightweight edge-based model achieves state-of-the-art performance on multiple visual CM-ReID benchmarks, while its cloud-based counterpart excels across all CM-ReID benchmarks. The MLLMEmbed-ReID framework thus presents a complete and effective solution for deploying unified MLLM-level intelligence on resource-constrained devices. The code and models will be open-sourced soon.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
