超越相关性：评估并改进检索器的视角感知能力
Beyond Relevance: Evaluate and Improve Retrievers on Perspective Awareness
AI总结：
本文构建多视角检索基准并发现现有检索器视角感知有限，进而提出基于表示空间几何特征的零样本投影方法，有效提升检索器的视角感知能力及下游任务性能。
摘要：
信息检索（IR）任务要求系统根据用户的信息需求识别相关文档。在现实场景中，检索器不仅需要依赖文档与查询之间的语义相关性，还应当识别用户查询背后微妙的意图或视角。例如，当被要求验证一个论断时，检索系统应当能够从支持与反对两种视角分别识别证据，以便下游系统做出公正的判断。在本工作中，我们研究检索器是否能够识别并响应查询中的不同视角——除了为某个论断找到相关文档之外，检索器能否区分支持性文档与反对性文档？我们改造并扩展了六个现有任务，构建了一个检索基准，其中除了中性的根查询之外，还包含以自由文本形式描述的多样化视角。我们表明，实验中涵盖的当前检索器对查询中细微不同的视角感知能力有限，并且还可能偏向某些特定视角。基于这一观察，我们进一步探索了利用检索器表示空间的几何特征，以零样本方式提升检索器视角感知能力的潜力。我们在同一组任务上证明了所提出的基于投影的方法的效率和有效性。进一步分析还表明，视角感知能力如何提升各种下游任务的性能：与非视角感知基线相比，在AmbigQA上准确率提高了4.2%，在论文写作任务上与指定观点的相关性提高了29.9%。
英文摘要：
The task of Information Retrieval (IR) requires a system to identify relevant documents based on users' information needs. In real-world scenarios, retrievers are expected to not only rely on the semantic relevance between the documents and the queries but also recognize the nuanced intents or perspectives behind a user query. For example, when asked to verify a claim, a retrieval system is expected to identify evidence from both supporting vs. contradicting perspectives, for the downstream system to make a fair judgment call. In this work, we study whether retrievers can recognize and respond to different perspectives of the queries -- beyond finding relevant documents for a claim, can retrievers distinguish supporting vs. opposing documents? We reform and extend six existing tasks to create a benchmark for retrieval, where we have diverse perspectives described in free-form text, besides root, neutral queries. We show that current retrievers covered in our experiments have limited awareness of subtly different perspectives in queries and can also be biased toward certain perspectives. Motivated by the observation, we further explore the potential to leverage geometric features of retriever representation space to improve the perspective awareness of retrievers in a zero-shot manner. We demonstrate the efficiency and effectiveness of our projection-based methods on the same set of tasks. Further analysis also shows how perspective awareness improves performance on various downstream tasks, with 4.2% higher accuracy on AmbigQA and 29.9% more correlation with designated viewpoints on essay writing, compared to non-perspective-aware baselines.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
