arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2309.07597cs.CLcs.AIcs.IR
收藏

C-Pack:用于通用中文嵌入的打包资源

C-Pack: Packed Resources For General Chinese Embeddings

Beijing Academy of AI(北京人工智能研究院);Renmin University of China(中国人民大学);HuggingFace;USTC(中国科学技术大学);University of Montreal(蒙特利尔大学)

Shitao Xiao, Zheng Liu, Peitian Zhang, Niklas Muennighoff, Defu Lian, Jian-Yun Nie

AI总结:

本文提出C-Pack资源包,包含C-MTEB基准、C-MTP数据集和C-TEM模型系列,显著提升通用中文嵌入性能,并同步发布达到SOTA的英文嵌入资源。

摘要:

我们介绍了C-Pack,这是一套显著推进通用中文嵌入领域的资源包。C-Pack包含三项关键资源。1) C-MTEB是一个全面的中文文本嵌入基准,涵盖6个任务和35个数据集。2) C-MTP是一个大规模文本嵌入数据集,从有标签和无标签的中文语料库中精心整理,用于训练嵌入模型。3) C-TEM是一系列涵盖多种规模的嵌入模型。在发布时,我们的模型在C-MTEB上超越所有先前的中文文本嵌入,提升幅度最高达+10%。我们还整合并优化了C-TEM的全套训练方法。伴随我们的通用中文嵌入资源,我们还发布了用于英文文本嵌入的数据和模型。英文模型在MTEB基准上达到最先进性能;同时,我们发布的英文数据比中文数据大2倍。所有这些资源均已在https://github.com/FlagOpen/FlagEmbedding公开发布。

英文摘要:

We introduce C-Pack, a package of resources that significantly advance the field of general Chinese embeddings. C-Pack includes three critical resources. 1) C-MTEB is a comprehensive benchmark for Chinese text embeddings covering 6 tasks and 35 datasets. 2) C-MTP is a massive text embedding dataset curated from labeled and unlabeled Chinese corpora for training embedding models. 3) C-TEM is a family of embedding models covering multiple sizes. Our models outperform all prior Chinese text embeddings on C-MTEB by up to +10% upon the time of the release. We also integrate and optimize the entire suite of training methods for C-TEM. Along with our resources on general Chinese embedding, we release our data and models for English text embeddings. The English models achieve state-of-the-art performance on MTEB benchmark; meanwhile, our released English data is 2 times larger than the Chinese data. All these resources are made publicly available at https://github.com/FlagOpen/FlagEmbedding.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑