Ceci n'est pas une explication: 评估语言学习系统中作为可解释性陷阱的解释失败
Ceci n'est pas une explication: Evaluating Explanation Failures as Explainability Pitfalls in Language Learning Systems
AI总结:
本文通过L2-Bench基准评估AI语言学习工具在六个关键维度上的反馈质量,指出表面有用但本质有缺陷的解释会导致可解释性陷阱,并分析语言学习环境如何放大这些风险。
摘要:
AI驱动的语言学习工具日益为全球数百万学习者提供即时、个性化的反馈。然而,这种反馈可能以学习者甚至教师难以察觉的方式失败,长期使用可能强化误解并侵蚀学习效果。我们提出了L2-Bench的一部分,这是一个用于评估语言教育中AI系统的基准,包括(但不限于)有效反馈的六个关键维度:诊断准确性、适当性意识、错误原因、优先级排序、改进指导和支持自我调节。我们分析了AI系统在这些维度上可能失败的方式。这些失败,我们认为会导致“可解释性陷阱”,即表面上看似有帮助但本质上有缺陷的AI生成解释,增加了成就、人机交互和社会情感危害的风险。我们讨论了语言学习的特定背景如何放大这些风险,并概述了在设计评估框架时我们认为值得更多关注的开放问题。我们的分析旨在扩展社区对可解释性陷阱类型学及其可能发生的上下文动态的理解,以鼓励AI开发者更好地设计安全、可信和有效的AI解释。
英文摘要:
AI-powered language learning tools increasingly provide instant, personalised feedback to millions of learners worldwide. However, this feedback can fail in ways that are difficult for learners--and even teachers--to detect, potentially reinforcing misconceptions and eroding learning outcomes over extended use. We present a portion of L2-Bench, a benchmark for evaluating AI systems in language education that includes (but is not limited to) six critical dimensions of effective feedback: diagnostic accuracy, awareness of appropriacy, causes of error, prioritisation, guidance for improvement, and supporting self-regulation. We analyse how AI systems can fail with respect to these dimensions. These failures, which we argue are conducive to "explainability pitfalls," are AI-generated explanations that appear helpful on the surface but are fundamentally flawed, increasing the risk of attainment, human-AI interaction, and socioaffective harms. We discuss how the specific context of language learning amplifies these risks and outline open questions we believe merit more attention when designing evaluation frameworks specifically. Our analysis aims to expand the community's understanding of both the typology of explainability pitfalls and the contextual dynamics in which they may occur in order to encourage AI developers to better design safe, trustworthy, and effective AI explanations.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。