TACO：通过任务映射引导的序列配置增强多模态上下文学习
TACO: Enhancing Multimodal In-context Learning via Task Mapping-Guided Sequence Configuration
AI总结：
本文从任务映射视角解释多模态 ICL，并提出轻量模型 TACO，利用任务感知注意力动态配置示例序列，在五个 LVLMs 和九个数据集上持续提升性能。
摘要：
多模态上下文学习（ICL）已成为利用大型视觉语言模型（LVLMs）能力的关键机制。然而，其有效性仍对输入 ICL 序列的质量高度敏感，尤其是在涉及复杂推理或开放式生成的任务中。一个主要局限在于，我们对 LVLMs 在推理过程中究竟如何利用这些序列理解有限。为弥合这一差距，我们系统地从任务映射的视角解释多模态 ICL，揭示示范内部及示范之间的局部与全局关系如何引导模型推理。基于这一洞见，我们提出 TACO，这是一种轻量级、基于 transformer 的模型，配备任务感知注意力，可动态配置 ICL 序列。通过将任务映射信号注入自回归解码过程，TACO 在序列构建与任务推理之间形成双向协同。在五个 LVLMs 和九个数据集上的实验表明，TACO 在多种 ICL 任务中持续优于基线方法。这些结果将任务映射确立为解释和改进多模态 ICL 的一个新颖且有价值的视角。
英文摘要：
Multimodal in-context learning (ICL) has emerged as a key mechanism for harnessing the capabilities of large vision-language models (LVLMs). However, its effectiveness remains highly sensitive to the quality of input ICL sequences, particularly for tasks involving complex reasoning or open-ended generation. A major limitation is our limited understanding of how LVLMs actually exploit these sequences during inference. To bridge this gap, we systematically interpret multimodal ICL through the lens of task mapping, which reveals how local and global relationships within and among demonstrations guide model reasoning. Building on this insight, we present TACO, a lightweight transformer-based model equipped with task-aware attention that dynamically configures ICL sequences. By injecting task-mapping signals into the autoregressive decoding process, TACO creates a bidirectional synergy between sequence construction and task reasoning. Experiments on five LVLMs and nine datasets demonstrate that TACO consistently surpasses baselines across diverse ICL tasks. These results position task mapping as a novel and valuable perspective for interpreting and improving multimodal ICL.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
