Hyper-ICL：基于双曲锚点蒸馏的注意力校准用于多模态上下文学习
Hyper-ICL: Attention Calibration with Hyperbolic Anchor Distillation for Multimodal In-Context Learning
AI总结：
提出Hyper-ICL，一种轻量级训练框架，通过低秩logit适配器和双曲锚点蒸馏损失校准注意力分布，无需推理时提供上下文示例即可重建演示效果，提升多模态上下文学习的准确性和稳定性。
摘要：
多模态上下文学习已成为多模态大语言模型的一种实用推理范式，其中少量交错的图像-文本上下文示例条件化模型以解决新任务。尽管灵活，但多模态ICL由于对演示格式、顺序和内容的敏感性，导致高推理延迟和不稳定性。为解决这些限制，我们提出Hyper-ICL，一种轻量级、基于训练的无演示多模态ICL框架，它直接在推理时无需ICD即可重建演示效果。Hyper-ICL学习一个参数高效的低秩logit级适配器，校准注意力分布以更好地匹配演示诱导的注意力重分布。为捕捉演示影响如何随查询变化，我们引入查询自适应调制机制，根据当前查询在层和头之间自适应控制token级的干预强度。最后，我们提出逐层双曲锚点蒸馏损失，通过Lorentz测地距离将中间学生特征对齐到演示条件化的教师。该损失鼓励学生重建ICD诱导的演示-查询关系。在六个不同多模态基准（包括VQAv2、OK-VQA和COCO Caption）上的大量实验表明，Hyper-ICL在准确性和稳定性上持续优于普通ICL和现有最先进方法。
英文摘要：
Multimodal In-Context Learning (ICL) has emerged as a practical inference paradigm for Multimodal Large Language Models, where a small set of interleaved image-text In-Context Demonstrations (ICDs) conditions the model to solve new tasks. Despite its flexibility, multimodal ICL incurs high inference latency and suffers from instability due to sensitivity to demonstration formatting, ordering, and content. To address these limitations, we propose Hyper-ICL, a lightweight, training-based framework for demonstration-free multimodal ICL that reconstructs demonstration effects directly without requiring ICDs at inference time. Hyper-ICL learns a parameter-efficient low-rank logit-level adapter that calibrates attention distributions to better match demonstration-induced attention redistribution. To capture how demonstration influence varies across queries, we introduce a query-adaptive modulation mechanism that adaptively controls intervention strength at token level across layers and heads based on the current query. Finally, we propose a layer-wise hyperbolic anchor distillation loss that aligns intermediate student features to a demonstration-conditioned teacher via Lorentz geodesic distance. This loss encourages the student to reconstruct the demonstration-query relationships induced by ICDs. Extensive experiments across six different multimodal benchmarks (including VQAv2, OK-VQA, and COCO Caption) demonstrate that Hyper-ICL consistently improves accuracy and stability over vanilla ICL and existing state-of-the-art methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
