arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2310.01232cs.LG
收藏

用于金融时间序列预测的模态感知Transformer

Modality-aware Transformer for Financial Time series Forecasting

IBM T.J. Watson Research Center(IBM TJ·沃森研究中心)

Hajar Emami, Xuan-Hong Dang, Yousaf Shah, Petros Zerfos

AI总结:

提出Modality-aware Transformer模型,通过特征级注意力及模态内/模态间/目标模态多头注意力机制融合文本与数值数据,有效提升金融时间序列预测精度。

摘要:

时间序列预测面临重大挑战,特别是当其准确性依赖于外部数据源而非仅依赖历史数值时。这一问题在金融领域尤为普遍,因为时间序列的未来行为通常与来自各种文本报告和众多经济指标的信息密切相关。在实践中,关键挑战在于构建一个可靠的时间序列预测模型,能够利用来自不同来源的数据并提取有价值的见解,以准确预测目标时间序列。在本研究中,我们应对这一挑战性问题,提出了一种新颖的基于多模态Transformer的模型,命名为Modality-aware Transformer。该模型擅长探索分类文本和数值时间序列的能力,通过其神经注意力机制提供见解,从而有效预测目标时间序列。为此,我们开发了特征级注意力层,促使模型关注每种数据模态内最相关的特征。通过引入所提出的特征级注意力,我们以将特征和时间注意力结合到MHA中的方式,开发了新颖的模态内多头注意力(MHA)、模态间MHA和目标模态MHA。这使MHA能够在考虑模态和特征重要性的情况下生成时间注意力,从而产生更具信息量的嵌入。所提出的模态感知结构使模型能够有效利用每种模态内的信息,并促进跨模态理解。我们在金融数据集上的广泛实验表明,Modality-aware Transformer优于现有方法,为多模态金融时间序列预测的复杂挑战提供了新颖且实用的解决方案。

英文摘要:

Time series forecasting presents a significant challenge, particularly when its accuracy relies on external data sources rather than solely on historical values. This issue is prevalent in the financial sector, where the future behavior of time series is often intricately linked to information derived from various textual reports and a multitude of economic indicators. In practice, the key challenge lies in constructing a reliable time series forecasting model capable of harnessing data from diverse sources and extracting valuable insights to predict the target time series accurately. In this work, we tackle this challenging problem and introduce a novel multimodal transformer-based model named the \textit{Modality-aware Transformer}. Our model excels in exploring the power of both categorical text and numerical timeseries to forecast the target time series effectively while providing insights through its neural attention mechanism. To achieve this, we develop feature-level attention layers that encourage the model to focus on the most relevant features within each data modality. By incorporating the proposed feature-level attention, we develop a novel Intra-modal multi-head attention (MHA), Inter-modal MHA and Target-modal MHA in a way that both feature and temporal attentions are incorporated in MHAs. This enables the MHAs to generate temporal attentions with consideration of modality and feature importance which leads to more informative embeddings. The proposed modality-aware structure enables the model to effectively exploit information within each modality as well as foster cross-modal understanding. Our extensive experiments on financial datasets demonstrate that Modality-aware Transformer outperforms existing methods, offering a novel and practical solution to the complex challenges of multi-modal financial time series forecasting.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑