arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.02304cs.AIcs.LG
收藏

比较解释并不足够,解释变化:需要新的标准来解释大型语言模型中的行为转变

Comparing Explanations is Not Enough, Explain the Change: New Standards are Needed to Explain Behavioral Shifts in Large Language Models

Scuola Normale Superiore(诺莱学院);ISTI-CNR(意大利国家研究委员会ISTI研究所);University of Pisa(比萨大学)

Martino Ciaperoni, Marzio Di Vece, Roberto Pellungrini, Luca Pappalardo, Fosca Giannotti, Francesco Giannini

AI总结:

本文提出了一种新的XAI方法,旨在解释大型语言模型在干预后行为转变的原因和机制,以应对现有解释方法无法解释行为转变的问题。

摘要:

大规模基础模型在受到缩放、微调、人类反馈强化学习或上下文学习等干预时会表现出行为转变。当前的可解释性方法结构上不适用于解释这些转变,因为它们要么将模型视为静态对象,如传统可解释AI(XAI)方法所做的,要么仅仅比较不同模型检查点的独立解释。因此,这些方法无法解释两个模型实例之间的功能转变,其中某种行为在干预后发生了变化。这种差距在欧盟人工智能法案、美国州立法和中国人工智能法规等司法管辖区中带来了重大治理风险,这些法规要求记录重大系统修改的因果链。本文主张,解释大型语言模型的行为转变需要一种系统的方法,将转变本身作为解释的主要对象:即解释干预如何和为何将参考模型转变为具有不同行为的更新模型。为了支持这一主张,我们引入了称为比较XAI(XAI_Δ)的新XAI范式,旨在解释两个模型检查点之间的差异,其中行为发生了变化,以及一组规范,规定XAI_Δ解释器和解释必须满足的条件,包括可比性、有效性、可操作性和监控,目标是将模型审计 grounded 在明确、可测量的要求中。最后,我们通过示例实验提供初步证据,表明在实践中需要XAI_Δ,将结果汇总成一份转换报告,直接可用于治理和事件记录。

英文摘要:

Large-scale foundation models exhibit behavioral shifts when subjected to interventions such as scaling, fine-tuning, reinforcement learning with human feedback, or in-context learning. Current explainability methods are structurally ill-suited to explain these shifts, because they either treat models as static objects, as traditional eXplainable AI (XAI) approaches do, or merely compare independent explanations across different checkpoints of a model. As a result, these approaches fail to explain the functional transition between two model instances in which a certain behavior has shifted following an intervention. This gap creates significant governance risks across jurisdictions including the EU AI Act, US state legislation, and Chinese AI regulations, which require documenting causal chains for substantial system modifications. This position paper argues that explaining behavioral shifts in large language models requires a principled approach that treats the shift itself as the primary object of explanation: namely, one that explains how and why an intervention transforms a reference model into an updated model with different behavior. To support this claim, we introduce Comparative XAI (XAI$_Δ$), a novel XAI paradigm aimed at explaining the difference between two model checkpoints where a behavior has shifted, together with a set of desiderata specifying what XAI$_Δ$ explainers and explanations must satisfy, including comparability, validity, actionability, and monitoring, with the goal of grounding model auditing in explicit, measurable requirements. Finally, we provide preliminary evidence suggesting the need for XAI$_Δ$ in practice through illustrative experiments, compiling the resulting findings into a transition report directly usable for governance and incident documentation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑