Vector-ICL：使用连续向量表示的上下文学习
Vector-ICL: In-context Learning with Continuous Vector Representations
AI总结：
本文提出 Vector-ICL，通过轻量级投影器将黑盒编码器产生的连续向量对齐到 LLM 嵌入空间，证明 LLM 可跨任务和模态进行向量上下文学习并取得优于少样本 ICL 等方法的效果。
摘要：
大型语言模型（LLM）已在文本数据上展现出显著的上下文学习（ICL）能力。我们探究这些能力能否扩展到来自不同领域、由黑盒预训练编码器获得的连续向量。通过轻量级投影器将输入数据与 LLM 的嵌入空间对齐，我们观察到 LLM 能够有效处理这些投影向量并从中学习，我们将其称为 Vector-ICL。具体而言，我们发现以通用语言建模目标预训练投影器即可实现 Vector-ICL，而面向任务的微调会进一步提升性能。在涵盖文本重建、数值函数回归、文本分类、摘要、分子描述生成、时间序列分类、图分类和 fMRI 解码等多种任务与模态的实验中，Vector-ICL 往往优于少样本 ICL 以及特定领域模型或调优方法。我们还进一步开展分析和案例研究，表明 LLM 具有超越传统基于词元的范式来处理向量表示的潜力。
英文摘要：
Large language models (LLMs) have shown remarkable in-context learning (ICL) capabilities on textual data. We explore whether these capabilities can be extended to continuous vectors from diverse domains, obtained from black-box pretrained encoders. By aligning input data with an LLM's embedding space through lightweight projectors, we observe that LLMs can effectively process and learn from these projected vectors, which we term Vector-ICL. In particular, we find that pretraining projectors with general language modeling objectives enables Vector-ICL, while task-specific finetuning further enhances performance. In our experiments across various tasks and modalities, including text reconstruction, numerical function regression, text classification, summarization, molecule captioning, time-series classification, graph classification, and fMRI decoding, Vector-ICL often surpasses both few-shot ICL and domain-specific model or tuning. We further conduct analyses and case studies, indicating the potential of LLMs to process vector representations beyond traditional token-based paradigms.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
