arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1809.03864cs.LGcs.AIcs.NEstat.ML
收藏

长短期记忆网络中审计细胞动力学的反应特征

Response Characterization for Auditing Cell Dynamics in Long Short-term Memory Networks

Massachusetts Institute of Technology (MIT)(麻省理工学院);Technische Universität Wien (TU Wien)(维也纳工业大学);Institute of Science and Technology (IST)(科学技术研究所)

Ramin M. Hasani, Alexander Amini, Mathias Lechner, Felix Naser, Radu Grosu, Daniela Rus

摘要:

在本文中,我们介绍了一种新的方法来解释循环神经网络(RNN),特别是长短期记忆网络(LSTM)在细胞水平。我们提出了一个系统的管道解释个人隐藏的状态动态网络内使用响应表征方法。通过分析一组解耦阶跃和正弦响应的可解释度量,计算单个单元对网络输出的排名贡献。因此,我们的方法能够唯一地识别具有深刻动态的神经元,通过消融分析量化动态特性和测试精度之间的关系,并解释网络容量对网络动态分布的影响。最后,我们通过评估一系列不同的基准序列数据集,证明了我们的方法的通用性和可扩展性。

英文摘要:

In this paper, we introduce a novel method to interpret recurrent neural networks (RNNs), particularly long short-term memory networks (LSTMs) at the cellular level. We propose a systematic pipeline for interpreting individual hidden state dynamics within the network using response characterization methods. The ranked contribution of individual cells to the network's output is computed by analyzing a set of interpretable metrics of their decoupled step and sinusoidal responses. As a result, our method is able to uniquely identify neurons with insightful dynamics, quantify relationships between dynamical properties and test accuracy through ablation analysis, and interpret the impact of network capacity on a network's dynamical distribution. Finally, we demonstrate generalizability and scalability of our method by evaluating a series of different benchmark sequential datasets.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑