arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2404.12283cs.CL
收藏

通过基于大型语言模型的文本丰富与重写提升嵌入性能

Enhancing Embedding Performance through Large Language Model-based Text Enrichment and Rewriting

Arizona State University(亚利桑那州立大学);MyAutoBio Inc.(迈欧托生物公司)

Nicholas Harris, Anand Butani, Syed Hashmy

AI总结:

本文提出利用大型语言模型在嵌入前对文本进行丰富和重写,以提升嵌入性能,在TwitterSemEval 2015数据集上取得显著改进,最佳得分达85.34。

摘要:

嵌入模型对于各种自然语言处理任务至关重要,但可能受到词汇量有限、缺乏上下文和语法错误等因素的限制。本文提出了一种新颖的方法,通过在嵌入过程之前利用大型语言模型(LLMs)来丰富和重写输入文本,从而提升嵌入性能。通过利用ChatGPT 3.5提供额外的上下文、纠正不准确之处并整合元数据,所提出的方法旨在增强嵌入模型的实用性和准确性。该方法的有效性在三个数据集上进行了评估:Banking77Classification、TwitterSemEval 2015和Amazon Counter-factual Classification。结果表明,在TwitterSemEval 2015数据集上,该方法相对于基线模型取得了显著改进,最佳提示在Massive Text Embedding Benchmark(MTEB)排行榜上得分达到85.34,而此前最佳得分为81.52。然而,在其他两个数据集上的表现则不那么突出,这凸显了考虑领域特定特征的重要性。研究结果表明,基于LLM的文本丰富在提升嵌入性能方面已显示出有前景的结果,尤其是在某些领域。因此,嵌入过程中的众多局限性可以被避免。

英文摘要:

Embedding models are crucial for various natural language processing tasks but can be limited by factors such as limited vocabulary, lack of context, and grammatical errors. This paper proposes a novel approach to improve embedding performance by leveraging large language models (LLMs) to enrich and rewrite input text before the embedding process. By utilizing ChatGPT 3.5 to provide additional context, correct inaccuracies, and incorporate metadata, the proposed method aims to enhance the utility and accuracy of embedding models. The effectiveness of this approach is evaluated on three datasets: Banking77Classification, TwitterSemEval 2015, and Amazon Counter-factual Classification. Results demonstrate significant improvements over the baseline model on the TwitterSemEval 2015 dataset, with the best-performing prompt achieving a score of 85.34 compared to the previous best of 81.52 on the Massive Text Embedding Benchmark (MTEB) Leaderboard. However, performance on the other two datasets was less impressive, highlighting the importance of considering domain-specific characteristics. The findings suggest that LLM-based text enrichment has shown promising results to improve embedding performance, particularly in certain domains. Hence, numerous limitations in the process of embedding can be avoided.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑