arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2207.06897cs.CL
收藏

小心商业化陷阱!当语言模型的可解释性与我们的语言心理模型相背离时

Beware the Rationalization Trap! When Language Model Explainability Diverges from our Mental Models of Language

University of Konstanz(康斯坦茨大学);ETH Zürich(苏黎世联邦理工学院)

Rita Sevastjanova, Mennatallah El-Assady

摘要:

语言模型学习和表示语言的方式与人类不同;它们学习的是形式,而不是意义。因此,为了评估语言模型可解释性的成功,我们需要考虑它与用户的语言心理模型的分歧的影响。在这份立场文件中,我们认为,为了避免有害的合理化和实现对语言模型的真实理解,解释过程必须满足三个主要条件:(1)解释必须真实地代表模型行为,即,具有高度的忠诚度;(2)解释必须完整,因为缺失的信息会扭曲真相;(3)解释必须考虑用户的心理模型,逐步验证一个人的知识并调整他们的理解。我们引入了一个决策树模型来展示当前解释未能达到其目标的潜在原因。我们进一步强调以人为本的设计需要从多个角度解释模型,逐步适应不断变化的用户期望的解释。

英文摘要:

Language models learn and represent language differently than humans; they learn the form and not the meaning. Thus, to assess the success of language model explainability, we need to consider the impact of its divergence from a user's mental model of language. In this position paper, we argue that in order to avoid harmful rationalization and achieve truthful understanding of language models, explanation processes must satisfy three main conditions: (1) explanations have to truthfully represent the model behavior, i.e., have a high fidelity; (2) explanations must be complete, as missing information distorts the truth; and (3) explanations have to take the user's mental model into account, progressively verifying a person's knowledge and adapting their understanding. We introduce a decision tree model to showcase potential reasons why current explanations fail to reach their objectives. We further emphasize the need for human-centered design to explain the model from multiple perspectives, progressively adapting explanations to changing user expectations.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑