首页/论文/1907.04670

arXiv 1907.046702019-07cs.LGcs.CLstat.COstat.ML

Improving the Performance of the LSTM and HMM Model via Hybridization

University of Toronto（多伦多大学）；University of Waterloo（滑铁卢大学）

Larkin Liu, Yu-Chung Lin, Joshua Reid

备注：: Working Manuscript

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

英文摘要相关论文

英文摘要：

Language models based on deep neural networks and traditional stochastic modelling have become both highly functional and effective in recent times. In this work, a general survey into the two types of language modelling is conducted. We investigate the effectiveness of the Hidden Markov Model (HMM), and the Long Short-Term Memory Model (LSTM). We analyze the hidden state structures common to both models, and present an analysis on structural similarity of the hidden states, common to both HMM's and LSTM's. We compare the LSTM's predictive accuracy and hidden state output with respect to the HMM for a varying number of hidden states. In this work, we justify that the less complex HMM can serve as an appropriate approximation of the LSTM model.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

Increasing the Interpretability of Recurrent Neural Networks Using Hidden Markov Models

Harvard University（哈佛大学）

As deep neural networks continue to revolutionize various application domains, there is increasing interest in making these powerful models more understandable and interpretable, and narrowing down the causes of good and bad predictions. We focus on recurrent neural networks (RNNs), state of the art models in speech recognition and translation. Our approach to increasing interpretability is by combining an RNN with a hidden Markov model (HMM), a simpler and more transparent model. We explore various combinations of RNNs and HMMs: an HMM trained on LSTM states; a hybrid model where an HMM is tr…

2016-09-30cs.CLcs.LGstat.ML

Increasing the Interpretability of Recurrent Neural Networks Using Hidden Markov Models

Harvard University（哈佛大学）

As deep neural networks continue to revolutionize various application domains, there is increasing interest in making these powerful models more understandable and interpretable, and narrowing down the causes of good and bad predictions. We focus on recurrent neural networks, state of the art models in speech recognition and translation. Our approach to increasing interpretability is by combining a long short-term memory (LSTM) model with a hidden Markov model (HMM), a simpler and more transparent model. We add the HMM state probabilities to the output layer of the LSTM, and then train the HMM…

2016-11-18cs.LGstat.ML

用于经济预测的 HMM-LSTM 融合模型

HMM-LSTM Fusion Model for Economic Forecasting

本文提出融合 HMM 隐状态与均值特征的 LSTM 经济预测模型，用于 CPI 通胀率预测，并结合 Integrated Gradients 提升可解释性；实验表明该方法能更准确捕捉复杂时间模式并减弱经济波动影响。

2025-01-01cs.LGecon.EMstat.ME

Superposition as Data Augmentation using LSTM and HMM in Small Training Sets

Novosibirsk State University（新西伯利亚国立大学）

Considering audio and image data as having quantum nature (data are represented by density matrices), we achieved better results on training architectures such as 3-layer stacked LSTM and HMM by mixing training samples using superposition augmentation and compared with plain default training and mix-up augmentation. This augmentation technique originates from the mix-up approach but provides more solid theoretical reasoning based on quantum properties. We achieved 3% improvement (from 68% to 71%) by using 38% lesser number of training samples in Russian audio-digits recognition task and 7,16% …

2019-10-24cs.LGeess.AS

Stock Market Trend Analysis Using Hidden Markov Model and Long Short Term Memory

Sun Yat-sen University（中山大学）

This paper intends to apply the Hidden Markov Model into stock market and and make predictions. Moreover, four different methods of improvement, which are GMM-HMM, XGB-HMM, GMM-HMM+LSTM and XGB-HMM+LSTM, will be discussed later with the results of experiment respectively. After that we will analyze the pros and cons of different models. And finally, one of the best will be used into stock market for timing strategy.

2021-04-20cs.LGq-fin.CPq-fin.PM

用于短期交通流量预测的混合隐马尔可夫 LSTM

Hybrid hidden Markov LSTM for short-term traffic flow prediction

The Pennsylvania State University（宾夕法尼亚州立大学）

本文针对短期交通流量预测，结合 HMM 的状态机制识别能力与 LSTM 的长期时序建模能力，提出混合 hidden Markov-LSTM，显著优于 Markov switching ARIMA 和 LSTM。

2023-07-17cs.LGstat.ML

更多相关论文收起更多论文

Implicit Language Model in LSTM for OCR

Information Sciences Institute（信息科学研究所）；University of Southern California（南加利福尼亚大学）

Neural networks have become the technique of choice for OCR, but many aspects of how and why they deliver superior performance are still unknown. One key difference between current neural network techniques using LSTMs and the previous state-of-the-art HMM systems is that HMM systems have a strong independence assumption. In comparison LSTMs have no explicit constraints on the amount of context that can be considered during decoding. In this paper we show that they learn an implicit LM and attempt to characterize the strength of the LM in terms of equivalent n-gram context. We show that this i…

2018-05-23cs.CVcs.LG

Fast and Accurate Recurrent Neural Network Acoustic Models for Speech Recognition

Google（谷歌）

We have recently shown that deep Long Short-Term Memory (LSTM) recurrent neural networks (RNNs) outperform feed forward deep neural networks (DNNs) as acoustic models for speech recognition. More recently, we have shown that the performance of sequence trained context dependent (CD) hidden Markov model (HMM) acoustic models using such LSTM RNNs can be equaled by sequence trained phone models initialized with connectionist temporal classification (CTC). In this paper, we present techniques that further improve performance of LSTM RNN acoustic models for large vocabulary speech recognition. We s…

2015-07-24cs.CLcs.LGcs.NE

利用统计学习和测试平台测量的IT入侵检测

IT Intrusion Detection Using Statistical Learning and Testbed Measurements

Dept. of Computer Science, KTH Royal Institute of Technology, Sweden（计算机科学系，皇家理工学院，瑞典）

本文提出利用统计学习方法，如HMM和LSTM，通过测试平台测量数据实现对IT入侵的检测，评估不同模型在预测攻击开始时间、类型和动作序列中的效果。

2025-12-22cs.CRcs.LG

融合 Capsule 与 LSTM 网络用于多变量数据无监督异常检测

Hybridization of Capsule and LSTM Networks for unsupervised anomaly detection on multivariate data

本文提出分支输入自编码器架构，融合 LSTM 与 Capsule Networks，以无监督方式检测多变量时间序列异常，显著缓解过拟合并在开源基准上达到最先进性能。

2022-02-11cs.LG

将 Persistent Homology 与神经网络混合用于时间序列预测：以波高预测为例

Hybridization of Persistent Homology with Neural Networks for Time-Series Prediction: A Case Study in Wave Height

本研究提出一种基于计算拓扑的特征工程方法，通过提取数据的拓扑特征并结合 FNNs、RNNs、LSTM 和 GRU 等神经网络模型，有效提升了波高时间序列预测的准确率并显著降低了误差。

2024-12-19cs.LGstat.ML

一种面向恶意软件分类的自然语言处理方法

A Natural Language Processing Approach to Malware Classification

San Jose State University（圣何塞州立大学）；Czech Technical University in Prague（布拉格捷克技术大学）

本研究提出一种结合隐马尔可夫模型与分类器的混合恶意软件分类方法，将HMM隐状态作为NLP风格特征，在恶意软件数据集上优于现有主流技术，其中HMM-随机森林效果最佳。

2023-07-07cs.CRcs.LG

Introducing the Hidden Neural Markov Chain framework

Telecom SudParis（南巴黎电信学院）；Institut Polytechnique de Paris（巴黎综合理工学院）；IBM GSB France（IBM法国全球业务咨询服务部）；Watson Department（沃森部门）

Nowadays, neural network models achieve state-of-the-art results in many areas as computer vision or speech processing. For sequential data, especially for Natural Language Processing (NLP) tasks, Recurrent Neural Networks (RNNs) and their extensions, the Long Short Term Memory (LSTM) network and the Gated Recurrent Unit (GRU), are among the most used models, having a "term-to-term" sequence processing. However, if many works create extensions and improvements of the RNN, few have focused on developing other ways for sequential data processing with neural networks in a "term-to-term" way. This…

2021-02-17cs.CLcs.LG

电话访谈中用于说话人日志的语言建模

Language Modelling for Speaker Diarization in Telephonic Interviews

Universitat Politecnica de Catalunya（加泰罗尼亚理工大学）

本文研究在电话访谈的说话人日志中结合语言与声学建模，提出基于LSTM和GMM的迭代分类算法，融合字符级词嵌入与声学分数，相比HMM/VB基线在词级DER上提升84.29%。

2025-01-28cs.LGcs.SDeess.AS