xVLM2vec：利用自知识蒸馏使基于 LVLM 的嵌入模型适配多语言性
xVLM2Vec: Adapting LVLM-based embedding models to multilinguality using Self-Knowledge Distillation
AI总结：
本文提出通过自知识蒸馏将英文训练的大型视觉-语言模型适配为多语言、多模态嵌入模型，并构建基准评估其有效性。
摘要：
在当前文献中，大多数嵌入模型都基于仅编码器 transformer 架构，以从给定输入（可以是文本、图像等）中提取稠密且有意义的表示。随着大语言模型的引入带来语言建模的最新进展，从这些经过大规模且广泛训练的模型中提取嵌入的可能性已得到探索。然而，当前研究聚焦于英文文本嵌入，而英文也是这些模型训练所使用的主要语言。此外，考虑多模态和多语言输入的模型非常少。鉴于此，我们提出了一种面向在英文数据上训练的大型视觉-语言模型的适配方法，以提升其提取多语言和多模态嵌入的性能。最后，我们设计并引入了一个基准，用于评估多语言和多模态嵌入模型的有效性。
英文摘要：
In the current literature, most embedding models are based on the encoder-only transformer architecture to extract a dense and meaningful representation of the given input, which can be a text, an image, and more. With the recent advances in language modeling thanks to the introduction of Large Language Models, the possibility of extracting embeddings from these large and extensively trained models has been explored. However, current studies focus on textual embeddings in English, which is also the main language on which these models have been trained. Furthermore, there are very few models that consider multimodal and multilingual input. In light of this, we propose an adaptation methodology for Large Vision-Language Models trained on English language data to improve their performance in extracting multilingual and multimodal embeddings. Finally, we design and introduce a benchmark to evaluate the effectiveness of multilingual and multimodal embedding models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
