大型语言模型如何理解相关性？一个机制可解释性视角
How do Large Language Models Understand Relevance? A Mechanistic Interpretability Perspective
AI总结：
本文从机制可解释性视角出发，利用激活修补技术揭示LLMs通过早期提取、中期处理和后期特定注意力头生成相关性判断的多阶段过程，为IR任务优化提供机制层面的启示。
摘要：
近期研究表明，大型语言模型（LLMs）能够评估相关性，并支持信息检索（IR）任务，如文档排序和相关性判断生成。然而，现成的LLMs理解和操作相关性的内部机制在很大程度上仍未得到探索。本文通过机制可解释性的视角，系统地研究不同LLM模块如何对相关性判断做出贡献。利用激活修补技术，我们分析了各种模型组件的作用，并识别出在生成逐点或逐对相关性判断时的一个多阶段、渐进式过程。具体而言，LLMs首先在早期层提取查询和文档信息，然后在中间层根据指令处理相关性信息，最后在后期层利用特定的注意力头以所需格式生成相关性判断。我们的发现为LLMs中相关性评估的底层机制提供了见解，并为未来利用LLMs进行IR任务的研究提供了有价值的启示。
英文摘要：
Recent studies have shown that large language models (LLMs) can assess relevance and support information retrieval (IR) tasks such as document ranking and relevance judgment generation. However, the internal mechanisms by which off-the-shelf LLMs understand and operationalize relevance remain largely unexplored. In this paper, we systematically investigate how different LLM modules contribute to relevance judgment through the lens of mechanistic interpretability. Using activation patching techniques, we analyze the roles of various model components and identify a multi-stage, progressive process in generating either pointwise or pairwise relevance judgment. Specifically, LLMs first extract query and document information in the early layers, then process relevance information according to instructions in the middle layers, and finally utilize specific attention heads in the later layers to generate relevance judgments in the required format. Our findings provide insights into the mechanisms underlying relevance assessment in LLMs, offering valuable implications for future research on leveraging LLMs for IR tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
