PUMA: 基于层剪枝的语言模型，用于具有模态自适应学习的高效统一多模态检索
PUMA: Layer-Pruned Language Model for Efficient Unified Multimodal Retrieval with Modality-Adaptive Learning
AI总结：
提出PUMA，通过层剪枝自蒸馏减少MLLM参数，并设计模态自适应对比学习损失（MAC-Loss）提升检索效率，在降低资源消耗的同时保持性能。
摘要：
随着多媒体内容的扩展，现实应用中对统一多模态检索（UMR）的需求日益增加。最近的工作利用多模态大语言模型（MLLM）来解决这一任务。然而，其庞大的参数量导致训练成本高、推理效率低。为此，我们提出PUMA：一种基于层剪枝的语言模型，用于具有模态自适应学习的高效统一多模态检索。我们的方法从结构和学习两个角度改进UMR。（1）在结构上，我们提出层剪枝自蒸馏，通过仅保留浅层来剪枝MLLM，同时从丢弃的深层中蒸馏特征作为教师信号。这减少了参数并保持了表示能力。（2）在学习方面，我们引入模态自适应对比学习损失（MAC-Loss），根据目标模态将批次内负样本分为更难的模态内和更易的模态间两组，分配不同的温度策略以增强学习效率。实验表明，我们的方法在保持强性能的同时显著减少了资源使用。
英文摘要：
As multimedia content expands, the demand for unified multimodal retrieval (UMR) in real-world applications increases. Recent work leverages multimodal large language models (MLLMs) to tackle this task. However, their large parameter size results in high training costs and low inference efficiency. To address this, we propose PUMA: a Layer-Pruned Language Model for Efficient Unified Multimodal Retrieval with Modality-Adaptive Learning. Our approach improves UMR from both structural and learning perspectives. (1) Structurally, we propose Layer-Pruned Self-Distillation, which prunes MLLMs by keeping only shallow layers while distilling features from dropped deep layers as teacher signals. This reduces parameters and preserves representation capability. (2) On the learning side, we introduce Modality-Adaptive Contrastive Learning Loss (MAC-Loss), which separates in-batch negatives into harder intra-modality and easier inter-modality groups based on the target modality, assigning different temperature strategies to enhance learning efficiency. Experiments show our method significantly reduces resource usage while maintaining strong performance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
