语言模型在学术检索中的低效性--一个实验性的走查
The Inefficiency of Language Models in Scholarly Retrieval: An Experimental Walk-through
摘要：
语言模型在人工智能驱动的科学IR系统中越来越受欢迎。本文评估了流行的科学语言模型在处理（i）短查询文本和（ii）文本邻居。我们的实验表明，即使在最宽松的条件下，也无法检索短查询文本的相关文档。此外，我们利用文本的邻居，产生的小扰动的原始文本，证明并不是所有的扰动导致嵌入空间中的近邻。此外，详尽的分类会产生几类拼写和语义相关的、部分相关的和完全不相关的邻居。检索性能的影响更大的表面形式，而不是语义的文本。
英文摘要：
Language models are increasingly becoming popular in AI-powered scientific IR systems. This paper evaluates popular scientific language models in handling (i) short-query texts and (ii) textual neighbors. Our experiments showcase the inability to retrieve relevant documents for a short-query text even under the most relaxed conditions. Additionally, we leverage textual neighbors, generated by small perturbations to the original text, to demonstrate that not all perturbations lead to close neighbors in the embedding space. Further, an exhaustive categorization yields several classes of orthographically and semantically related, partially related, and completely unrelated neighbors. Retrieval performance turns out to be more influenced by the surface form rather than the semantics of the text.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
