用于解释国际象棋智能体规划的对比稀疏自编码器
Contrastive Sparse Autoencoders for Interpreting Planning of Chess-Playing Agents
AI总结：
针对黑盒国际象棋智能体难以解释多步规划的问题，提出对比稀疏自编码器（CSAE），通过对比成对博弈轨迹提取规划相关概念，并结合特征分类与健全性检查验证解释质量。
摘要：
AI 将国际象棋系统提升到了超人类水平，但这些系统严重依赖黑盒算法。这在确保终端用户透明度方面是不可持续的，尤其是当这些系统负责敏感决策时。近期的可解释性研究表明，深度神经网络（DNN）的内部表征是可以理解的，并包含人类可理解的概念。然而，这些方法很少结合情境，而且通常基于单一隐藏状态，这使其无法解释多步推理，例如规划。为此，我们提出对比稀疏自编码器（CSAE），这是一种用于研究成对博弈轨迹的新颖框架。借助 CSAE，我们能够提取并解释对国际象棋智能体规划有意义的概念。我们首先重点对 CSAE 特征进行定性分析，随后提出自动化的特征分类法。此外，为评估所训练 CSAE 的质量，我们设计了健全性检查，以排除结果中的虚假相关性。
英文摘要：
AI led chess systems to a superhuman level, yet these systems heavily rely on black-box algorithms. This is unsustainable in ensuring transparency to the end-user, particularly when these systems are responsible for sensitive decision-making. Recent interpretability work has shown that the inner representations of Deep Neural Networks (DNNs) were fathomable and contained human-understandable concepts. Yet, these methods are seldom contextualised and are often based on a single hidden state, which makes them unable to interpret multi-step reasoning, e.g. planning. In this respect, we propose contrastive sparse autoencoders (CSAE), a novel framework for studying pairs of game trajectories. Using CSAE, we are able to extract and interpret concepts that are meaningful to the chess-agent plans. We primarily focused on a qualitative analysis of the CSAE features before proposing an automated feature taxonomy. Furthermore, to evaluate the quality of our trained CSAE, we devise sanity checks to wave spurious correlations in our results.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
