FUSE 语言模型：跨分词器的零样本适配器发现以优化提示词
FUSE-ing Language Models: Zero-Shot Adapter Discovery for Prompt Optimization Across Tokenizers
AI总结：
针对不同分词器导致提示词知识迁移困难的问题，本文提出 FUSE，用三阶张量对齐语义嵌入并近似跨模型梯度，实现零样本适配器发现，且在图像描述任务中验证有效。
摘要：
大语言模型的广泛使用产生了大量分词器和嵌入空间，使提示词发现任务中的知识迁移变得困难。本文提出 FUSE（Flexible Unification of Semantic Embeddings，语义嵌入灵活统一），这是一种低成本方法，用于近似一个适配器层，把一个模型的文本嵌入空间映射到另一个模型，即使二者使用不同分词器也可实现。作者引入了基于三阶张量的模型嵌入空间表示，用于对齐因不同分词器而被拆分的语义嵌入，并利用该表示推导一个模型输出相对于另一模型嵌入空间的梯度近似。通过在视觉-语言模型和因果语言模型上针对图像描述生成及基于情感的图像描述生成开展多目标优化，作者证明了该方法的有效性。
英文摘要：
The widespread use of large language models has resulted in a multitude of tokenizers and embedding spaces, making knowledge transfer in prompt discovery tasks difficult. In this work, we propose FUSE (Flexible Unification of Semantic Embeddings), an inexpensive approach to approximating an adapter layer that maps from one model's textual embedding space to another, even across different tokenizers. We introduce a third-order tensor-based representation of a model's embedding space that aligns semantic embeddings that have been split apart by different tokenizers, and use this representation to derive an approximation of the gradient of one model's outputs with respect to another model's embedding space. We show the efficacy of our approach via multi-objective optimization over vision-language and causal language models for image captioning and sentiment-based image captioning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
