为何大语言模型可以秘密超越嵌入相似性在信息检索中的表现
Why Large Language Models can Secretly Outperform Embedding Similarity in Information Retrieval
AI总结：
本文探讨了大语言模型在信息检索中通过推理超越传统嵌入相似性方法的潜力，并指出当前注释数据集难以准确评估其效果。
摘要：
随着大语言模型（LLMs）的出现，信息检索领域出现了新的方法，其中相关性是通过语言理解与推理直接估计的，而不是通过嵌入相似性。我们主张相似性是对相关性的短视解释，而基于LLM的相关性判断系统（LLM-RJS）（具有推理）有潜力通过克服这一限制来超越神经嵌入检索系统（NERS）。使用TREC-DL 2019段落检索数据集，我们比较了各种LLM-RJS与NERS，但观察到没有显著的改进。随后，我们通过比较具有和没有推理的LLM-RJS来分析推理的影响。我们发现人类注释也受短视影响，而推理LLM-RJS中的假阳性主要是由于注释中的短视错误。我们得出结论：LLM-RJS确实有能力解决NERS中的短视限制，但无法通过标准注释的相关性数据集来评估。
英文摘要：
With the emergence of Large Language Models (LLMs), new methods in Information Retrieval are available in which relevance is estimated directly through language understanding and reasoning, instead of embedding similarity. We argue that similarity is a short-sighted interpretation of relevance, and that LLM-Based Relevance Judgment Systems (LLM-RJS) (with reasoning) have potential to outperform Neural Embedding Retrieval Systems (NERS) by overcoming this limitation. Using the TREC-DL 2019 passage retrieval dataset, we compare various LLM-RJS with NERS, but observe no noticeable improvement. Subsequently, we analyze the impact of reasoning by comparing LLM-RJS with and without reasoning. We find that human annotations also suffer from short-sightedness, and that false-positives in the reasoning LLM-RJS are primarily mistakes in annotations due to short-sightedness. We conclude that LLM-RJS do have the ability to address the short-sightedness limitation in NERS, but that this cannot be evaluated with standard annotated relevance datasets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
