首页/论文/2012.06954

arXiv 2012.069542020-12-13cs.LGcs.AI

MEME: Generating RNN Model Explanations via Model Extraction

University of Cambridge（剑桥大学）

Dmitry Kazhdan, Botty Dimanov, Mateja Jamnik, Pietro Liò

备注：: Presented at the HAMLETS workshop at the 34th Conference on Neural Information Processing Systems (NeurIPS 2020)

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

英文摘要相关论文

英文摘要：

Recurrent Neural Networks (RNNs) have achieved remarkable performance on a range of tasks. A key step to further empowering RNN-based approaches is improving their explainability and interpretability. In this work we present MEME: a model extraction approach capable of approximating RNNs with interpretable models represented by human-understandable concepts and their interactions. We demonstrate how MEME can be applied to two multivariate, continuous data case studies: Room Occupation Prediction, and In-Hospital Mortality Prediction. Using these case-studies, we show how our extracted models can be used to interpret RNNs both locally and globally, by approximating RNN decision-making via interpretable concept interactions.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

DeepCover：使用状态机提取推进RNN测试覆盖与在线错误预测

DeepCover: Advancing RNN Test Coverage and Online Error Prediction using State Machine Extraction

University of Tehran（德黑兰大学）

本文提出从RNN模型中提取状态机（SM）以提高可解释性，并基于SM引入六个模型覆盖准则和树模型进行在线错误预测，在MNIST和Mini Speech Commands数据集上AUC超过80%。

2024-02-10cs.LG

An Empirical Evaluation of Rule Extraction from Recurrent Neural Networks

McGill University（麦吉尔大学）；Pennsylvania State University（宾夕法尼亚州立大学）

Rule extraction from black-box models is critical in domains that require model validation before implementation, as can be the case in credit scoring and medical diagnosis. Though already a challenging problem in statistical learning in general, the difficulty is even greater when highly non-linear, recursive models, such as recurrent neural networks (RNNs), are fit to data. Here, we study the extraction of rules from second-order recurrent neural networks trained to recognize the Tomita grammars. We show that production rules can be stably extracted from trained RNNs and that in certain case…

2018-11-14cs.LG

Model Extraction Attacks against Recurrent Neural Networks

Osaka University（大阪大学）

Model extraction attacks are a kind of attacks in which an adversary obtains a new model, whose performance is equivalent to that of a target model, via query access to the target model efficiently, i.e., fewer datasets and computational resources than those of the target model. Existing works have dealt with only simple deep neural networks (DNNs), e.g., only three layers, as targets of model extraction attacks, and hence are not aware of the effectiveness of recurrent neural networks (RNNs) in dealing with time-series data. In this work, we shed light on the threats of model extraction attac…

2020-02-01cs.CRcs.LG

DOER: Dual Cross-Shared RNN for Aspect Term-Polarity Co-Extraction

Southwest Jiaotong University（西南交通大学）；University of Illinois at Chicago（伊利诺伊大学芝加哥分校）；JD Intelligent Cities Business Unit（京东智能城市事业部）；JD Intelligent Cities Research（京东智能城市研究院）

This paper focuses on two related subtasks of aspect-based sentiment analysis, namely aspect term extraction and aspect sentiment classification, which we call aspect term-polarity co-extraction. The former task is to extract aspects of a product or service from an opinion document, and the latter is to identify the polarity expressed in the document about these extracted aspects. Most existing algorithms address them as two separate tasks and solve them one by one, or only perform one task, which can be complicated for real applications. In this paper, we treat these two tasks as two sequence…

2019-06-05cs.CLcs.LG

Extracting Finite Automata from RNNs Using State Merging

Center for Data Science, NYU（纽约大学数据科学中心）

One way to interpret the behavior of a blackbox recurrent neural network (RNN) is to extract from it a more interpretable discrete computational model, like a finite state machine, that captures its behavior. In this work, we propose a new method for extracting finite automata from RNNs inspired by the state merging paradigm from grammatical inference. We demonstrate the effectiveness of our method on the Tomita languages benchmark, where we find that it is able to extract faithful automata from RNNs trained on all languages in the benchmark. We find that extraction performance is aided by the…

2022-04-14cs.LG

用于自然语言任务的循环神经网络的加权自动机提取与解释

Weighted Automata Extraction and Explanation of Recurrent Neural Networks for Natural Language Tasks

Peking University（北京大学）；University of Oxford（牛津大学）

提出WFA提取与解释框架，解决RNN在自然语言任务中的转移稀疏和上下文丢失问题，提升提取精度并实现基于TME的面向任务的解释。

2023-06-24cs.CLcs.LG

更多相关论文收起更多论文

Extracting Weighted Finite Automata from Recurrent Neural Networks for Natural Languages

Peking University（北京大学）

Recurrent Neural Networks (RNNs) have achieved tremendous success in sequential data processing. However, it is quite challenging to interpret and verify RNNs' behaviors directly. To this end, many efforts have been made to extract finite automata from RNNs. Existing approaches such as exact learning are effective in extracting finite-state models to characterize the state dynamics of RNNs for formal languages, but are limited in the scalability to process natural languages. Compositional approaches that are scablable to natural languages fall short in extraction precision. In this paper, we i…

2022-09-27cs.CLcs.LG

KPI-BERT: A Joint Named Entity Recognition and Relation Extraction Model for Financial Reports

Fraunhofer IAIS（弗劳恩霍夫智能分析与信息系统研究所）；University of Bonn（波恩大学）；PricewaterhouseCoopers GmbH（普华永道有限责任公司）

We present KPI-BERT, a system which employs novel methods of named entity recognition (NER) and relation extraction (RE) to extract and link key performance indicators (KPIs), e.g. "revenue" or "interest expenses", of companies from real-world German financial documents. Specifically, we introduce an end-to-end trainable architecture that is based on Bidirectional Encoder Representations from Transformers (BERT) combining a recurrent neural network (RNN) with conditional label masking to sequentially tag entities before it classifies their relations. Our model also introduces a learnable RNN-b…

2022-08-03cs.AIcs.CLcs.LG

Understanding Medical Conversations: Rich Transcription, Confidence Scores & Information Extraction

Google（谷歌）

In this paper, we describe novel components for extracting clinically relevant information from medical conversations which will be available as Google APIs. We describe a transformer-based Recurrent Neural Network Transducer (RNN-T) model tailored for long-form audio, which can produce rich transcriptions including speaker segmentation, speaker role labeling, punctuation and capitalization. On a representative test set, we compare performance of RNN-T models with different encoders, units and streaming constraints. Our transformer-based streaming model performs at about 20% WER on the ASR tas…

2021-04-06cs.LG

Weighted Automata Extraction from Recurrent Neural Networks via Regression on State Spaces

National Institute of Informatics（国立信息学研究所）；The Graduate University for Advanced Studies（综合研究大学院大学）

We present a method to extract a weighted finite automaton (WFA) from a recurrent neural network (RNN). Our algorithm is based on the WFA learning algorithm by Balle and Mohri, which is in turn an extension of Angluin's classic \lstar algorithm. Our technical novelty is in the use of \emph{regression} methods for the so-called equivalence queries, thus exploiting the internal state space of an RNN to prioritize counterexample candidates. This way we achieve a quantitative/weighted extension of the recent work by Weiss, Goldberg and Yahav that extracts DFAs. We experimentally evaluate the accur…

2019-11-20cs.LGstat.ML

Generative Choreography using Deep Learning

Peltarion（Peltarion 公司）；The Lulu Art Group（Lulu 艺术集团）

Recent advances in deep learning have enabled the extraction of high-level features from raw sensor data which has opened up new possibilities in many different fields, including computer generated choreography. In this paper we present a system chor-rnn for generating novel choreographic material in the nuanced choreographic language and style of an individual choreographer. It also shows promising results in producing a higher level compositional cohesion, rather than just generating sequences of movement. At the core of chor-rnn is a deep recurrent neural network trained on raw motion captu…

2016-05-23cs.AIcs.LGcs.MM

Chipmunk: A Systolically Scalable 0.9 mm${}^2$, 3.08 Gop/s/mW @ 1.2 mW Accelerator for Near-Sensor Recurrent Neural Network Inference

ETH Zurich（苏黎世联邦理工学院）；University of Bologna（博洛尼亚大学）

Recurrent neural networks (RNNs) are state-of-the-art in voice awareness/understanding and speech recognition. On-device computation of RNNs on low-power mobile and wearable devices would be key to applications such as zero-latency voice-based human-machine interfaces. Here we present Chipmunk, a small (<1 mm${}^2$) hardware accelerator for Long-Short Term Memory RNNs in UMC 65 nm technology capable to operate at a measured peak efficiency up to 3.08 Gop/s/mW at 1.24 mW peak power. To implement big RNN models without incurring in huge memory transfer overhead, multiple Chipmunk engines can coo…

2018-02-20cs.DCcs.LGcs.NE

权重到代码：从离散Transformer中提取可解释算法

Weights to Code: Extracting Interpretable Algorithms from the Discrete Transformer

Key Laboratory of High Confidence Software Technology (PKU), MOE, Beijing, China（高可信软件技术重点实验室（PKU），教育部，北京，中国）；School of Computer Science, Peking University, Beijing, China（计算机学院，北京大学，北京，中国）；Shanghai AI Lab, Shanghai, China（上海人工智能实验室，上海，中国）；Shanghai Innovation Institute, Shanghai, China（上海创新研究院，上海，中国）

提出离散Transformer架构，通过温度退火采样注入离散性，结合假设检验和符号回归从模型权重中提取可解释算法，在离散任务上性能与RNN基线相当，并扩展到连续中间计算任务。

2026-05-29cs.CLcs.LG