生成式表示指令微调
Generative Representational Instruction Tuning
AI总结：
本文提出生成式表示指令微调（GRIT），通过指令区分任务训练大型语言模型同时处理生成与嵌入任务，在多项基准上达到先进水平并无性能损失地统一两类任务，显著加速检索增强生成。
摘要：
所有基于文本的语言问题都可以归结为生成或嵌入。当前的模型仅在其中一项上表现良好。我们引入了生成式表示指令微调（GRIT），通过指令区分生成和嵌入任务，从而训练大型语言模型同时处理这两类任务。与其他开放模型相比，我们得到的 GritLM 7B 在 Massive Text Embedding Benchmark (MTEB) 上创造了新的技术水平，并在一系列生成任务中超越了所有同等规模及以下的模型。通过进一步扩大规模，GritLM 8x7B 超越了我们尝试过的所有开放生成式语言模型，同时仍位列最佳嵌入模型之中。值得注意的是，我们发现 GRIT 与仅在生成或嵌入数据上训练的效果相当，因此我们可以在无性能损失的情况下统一两者。除其他益处外，通过 GRIT 的统一使得长文档的检索增强生成（RAG）速度提升了 > 60%，不再需要单独的检索和生成模型。模型、代码等可在 https://github.com/ContextualAI/gritlm 免费获取。
英文摘要：
All text-based language problems can be reduced to either generation or embedding. Current models only perform well at one or the other. We introduce generative representational instruction tuning (GRIT) whereby a large language model is trained to handle both generative and embedding tasks by distinguishing between them through instructions. Compared to other open models, our resulting GritLM 7B sets a new state of the art on the Massive Text Embedding Benchmark (MTEB) and outperforms all models up to its size on a range of generative tasks. By scaling up further, GritLM 8x7B outperforms all open generative language models that we tried while still being among the best embedding models. Notably, we find that GRIT matches training on only generative or embedding data, thus we can unify both at no performance loss. Among other benefits, the unification via GRIT speeds up Retrieval-Augmented Generation (RAG) by > 60% for long documents, by no longer requiring separate retrieval and generation models. Models, code, etc. are freely available at https://github.com/ContextualAI/gritlm.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
