上下文利用的高亮解释评估框架
Evaluation Framework for Highlight Explanations of Context Utilisation in Language Models
AI总结:
本文提出首个高亮解释评估框架,用于评估语言模型中上下文利用的解释有效性,通过可控测试案例验证了MechLight在不同场景中的优越性,发现现有方法在长上下文和位置偏倚上存在挑战。
摘要:
上下文利用,即语言模型在生成响应时整合相关上下文信息的能力,仍对用户来说是模糊的,用户无法确定模型是依赖参数记忆还是提供的上下文,也无法识别具体哪些上下文片段影响响应。高亮解释(HEs)提供了一种自然的解决方案,可以指向确切的上下文片段和令牌,影响模型输出。然而,现有工作未评估其在准确解释上下文利用方面的有效性。本文通过引入首个黄金标准HE评估框架来填补这一空白,使用具有已知真实上下文使用的受控测试案例,避免了现有间接代理评估的局限性。为了展示框架的广泛适用性,我们评估了四种HE方法——三种已建立的技术和MechLight,一种我们为此任务改编的机械可解释性方法——在四个上下文场景、四个数据集和五个LM上。总体而言,我们发现MechLight在所有上下文场景中表现最佳。然而,所有方法在长上下文和位置偏倚上都遇到困难,指出了解释准确性中的根本挑战,需要新的方法来在大规模上提供可靠的上下文利用解释。
英文摘要:
Context utilisation, the ability of Language Models (LMs) to incorporate relevant information from the provided context when generating responses, remains largely opaque to users, who cannot determine whether models draw from parametric memory or provided context, nor identify which specific context pieces inform the response. Highlight explanations (HEs) offer a natural solution as they can point the exact context pieces and tokens that influenced model outputs. However, no existing work evaluates their effectiveness in accurately explaining context utilisation. We address this gap by introducing the first gold standard HE evaluation framework for context attribution, using controlled test cases with known ground-truth context usage, which avoids the limitations of existing indirect proxy evaluations. To demonstrate the framework's broad applicability, we evaluate four HE methods -- three established techniques and MechLight, a mechanistic interpretability approach we adapt for this task -- across four context scenarios, four datasets, and five LMs. Overall, we find that MechLight performs best across all context scenarios. However, all methods struggle with longer contexts and exhibit positional biases, pointing to fundamental challenges in explanation accuracy that require new approaches to deliver reliable context utilisation explanations at scale.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。