arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2203.13787stat.MLcs.LGeess.SP
收藏

面向端到端优化的序列数据预测混合框架

A Hybrid Framework for Sequential Data Prediction with End-to-End Optimization

Bilkent University(比尔肯特大学)

Mustafa E. Aydın, Suleyman S. Kozat

AI总结:

本文提出一种端到端混合框架,将LSTM递归特征提取与soft GBDT梯度提升回归相结合,通过随机梯度下降联合优化,有效缓解传统非线性预测对手工特征和模型选择的需求,并在多个真实数据集上取得显著性能提升。

摘要:

我们研究在线环境中的非线性预测问题,并提出一种混合模型,通过端到端架构有效缓解传统非线性预测/回归方法对手工设计特征和人工模型选择的需求。具体而言,我们使用递归结构从序列信号中提取特征,同时保留状态信息(即历史信息),并使用提升决策树生成最终输出。该连接以端到端方式进行,我们使用随机梯度下降联合优化整个架构,并为此提供反向传播更新方程。特别地,我们采用循环神经网络(LSTM)对序列数据进行自适应特征提取,并采用梯度提升机制(soft GBDT)进行有效的监督回归。我们的框架具有通用性,只要其他深度学习架构(如RNN和GRU)以及用于决策的机器学习算法是可微的,就可以用于特征提取。我们在合成数据上展示了算法的学习行为,并在各种真实数据集上展示了相对于传统方法的显著性能提升。此外,我们公开分享所提出方法的源代码,以促进进一步研究。

英文摘要:

We investigate nonlinear prediction in an online setting and introduce a hybrid model that effectively mitigates, via an end-to-end architecture, the need for hand-designed features and manual model selection issues of conventional nonlinear prediction/regression methods. In particular, we use recursive structures to extract features from sequential signals, while preserving the state information, i.e., the history, and boosted decision trees to produce the final output. The connection is in an end-to-end fashion and we jointly optimize the whole architecture using stochastic gradient descent, for which we also provide the backward pass update equations. In particular, we employ a recurrent neural network (LSTM) for adaptive feature extraction from sequential data and a gradient boosting machinery (soft GBDT) for effective supervised regression. Our framework is generic so that one can use other deep learning architectures for feature extraction (such as RNNs and GRUs) and machine learning algorithms for decision making as long as they are differentiable. We demonstrate the learning behavior of our algorithm on synthetic data and the significant performance improvements over the conventional methods over various real life datasets. Furthermore, we openly share the source code of the proposed method to facilitate further research.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑