ReAttn：通过注意力重新加权改进基于注意力的重排序
ReAttn: Improving Attention-based Re-ranking via Attention Re-weighting
AI总结：
ReAttn通过注意力重新加权策略，改进基于注意力的重排序方法，减少词法偏差并提升重排序效果。
摘要：
近期大型语言模型（LLMs）的强大能力使其在零样本重排序任务中表现出色。基于注意力的重排序方法通过直接从注意力权重中推导相关性分数，为基于生成的重排序方法提供了一种高效且可解释的替代方案。然而，它们仍面临两个主要限制。首先，注意力信号高度集中在少数文档中的少量令牌上，使其他令牌变得无法区分。其次，注意力经常过度强调与查询语义相似的短语，导致无关文档仅因词法相似而被误判为相关。在本文中，我们提出了ReAttn，一种针对基于注意力的重排序方法的后处理重新加权策略。它首先计算跨文档IDF加权，以降低在候选文档中频繁出现的查询重叠令牌的注意力权重，从而减少词法偏差并强调独特术语。然后，它采用基于熵的正则化来缓解注意力过度集中，鼓励在信息性令牌上实现更平衡的分布。这两种调整直接作用于现有的注意力权重，无需额外的训练或监督。广泛的实验表明了我们方法的有效性。
英文摘要：
The strong capabilities of recent Large Language Models (LLMs) have made them highly effective for zero-shot re-ranking task. Attention-based re-ranking methods, which derive relevance scores directly from attention weights, offer an efficient and interpretable alternative to generation-based re-ranking methods. However, they still face two major limitations. First, attention signals are highly concentrated a small subset of tokens within a few documents, making others indistinguishable. Second, attention often overemphasizes phrases lexically similar to the query, yielding biased rankings that irrelevant documents with mere lexical resemblance are regarded as relevant. In this paper, we propose \textbf{ReAttn}, a post-hoc re-weighting strategy for attention-based re-ranking methods. It first compute the cross-document IDF weighting to down-weight attention on query-overlapping tokens that frequently appear across the candidate documents, reducing lexical bias and emphasizing distinctive terms. It then employs entropy-based regularization to mitigate over-concentrated attention, encouraging a more balanced distribution across informative tokens. Both adjustments operate directly on existing attention weights without additional training or supervision. Extensive experiments demonstrate the effectiveness of our method.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
