打造知识：探索基于聊天的搜索引擎的创造机制
Crafting Knowledge: Exploring the Creative Mechanisms of Chat-Based Search Engines
AI总结：
本文通过采集 Bing Chat、传统搜索及基于 GPT-4 的 RAG API 的引用网站数据，揭示聊天搜索引擎偏好可读、结构化、低困惑度文本，且该偏好源自底层 LLM。
摘要：
在数字信息传播领域，搜索引擎是连接信息寻求者与提供者的关键渠道。以 Bing Chat 为代表、利用大语言模型（LLM）和检索增强生成（RAG）的基于聊天的搜索引擎的出现，标志着搜索生态系统的一次进化飞跃。它们在解读网络信息并以类人的理解和创造力生成回答方面展现出元认知能力。然而，LLM 的复杂本质使其“认知”过程并不透明，甚至令其设计者也难以理解。本研究旨在剖析由 LLM 驱动的基于聊天的搜索引擎（具体为 Bing Chat）为其回答选择信息来源的机制。为此，研究通过与 New Bing 的交互编制了一个大规模数据集，记录了它引用的网站以及传统搜索引擎列出的网站。借助自然语言处理（NLP）技术，研究发现 Bing Chat 偏好的内容不仅具有可读性和形式化结构，而且表现出较低的困惑度，这表明其独特地倾向于底层 LLM 可预测的文本。为进一步丰富分析，我们通过与基于 GPT-4 的知识检索 API 交互获取了另一个数据集，揭示了该 RAG API 与 Bing Chat 之间一致的文本偏好。这种共识表明，这些文本偏好本质上源于底层语言模型，而非由 Bing Chat 的开发者显式打造。此外，我们的调查记录显示，与传统搜索引擎排名最高的网站相比，RAG 技术引用的网站之间具有更高的相似性。
英文摘要：
In the domain of digital information dissemination, search engines act as pivotal conduits linking information seekers with providers. The advent of chat-based search engines utilizing Large Language Models (LLMs) and Retrieval Augmented Generation (RAG), exemplified by Bing Chat, marks an evolutionary leap in the search ecosystem. They demonstrate metacognitive abilities in interpreting web information and crafting responses with human-like understanding and creativity. Nonetheless, the intricate nature of LLMs renders their "cognitive" processes opaque, challenging even their designers' understanding. This research aims to dissect the mechanisms through which an LLM-powered chat-based search engine, specifically Bing Chat, selects information sources for its responses. To this end, an extensive dataset has been compiled through engagements with New Bing, documenting the websites it cites alongside those listed by the conventional search engine. Employing natural language processing (NLP) techniques, the research reveals that Bing Chat exhibits a preference for content that is not only readable and formally structured, but also demonstrates lower perplexity levels, indicating a unique inclination towards text that is predictable by the underlying LLM. Further enriching our analysis, we procure an additional dataset through interactions with the GPT-4 based knowledge retrieval API, unveiling a congruent text preference between the RAG API and Bing Chat. This consensus suggests that these text preferences intrinsically emerge from the underlying language models, rather than being explicitly crafted by Bing Chat's developers. Moreover, our investigation documents a greater similarity among websites cited by RAG technologies compared to those ranked highest by conventional search engines.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
