arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2102.06828cs.LGstat.ML
收藏

基于注意力共享的时间序列预测领域自适应

Domain Adaptation for Time Series Forecasting via Attention Sharing

AWS AI Labs(AWS人工智能实验室);Rutgers University(罗格斯大学)

Xiaoyong Jin, Youngsuk Park, Danielle C. Maddix, Hao Wang, Yuyang Wang

摘要:

最近,深度神经网络在时间序列预测领域越来越受欢迎。他们成功的主要原因是他们能够有效地捕捉跨多个相关时间序列的复杂时间动态。这些深度预测的优势只有在有足够数据的情况下才开始显现。这对实践中的典型预测问题提出了挑战,其中存在有限数量的时间序列或每个时间序列的观测值,或两者兼而有之。为了解决这一数据稀缺问题,我们提出了一种新的领域自适应框架,领域自适应预测器。该方法利用来自具有丰富数据样本的相关领域(源)的统计优势,以提高具有有限数据的感兴趣领域(目标)的性能。特别是,我们使用了一个基于注意力的共享模块与域跨域和私人模块的各个域。我们诱导域不变的潜在特征(查询和键),并同时重新训练特定于域的特征(值),以实现对源域和目标域的预测器的联合训练。一个主要的见解是,我们的对齐键的设计允许目标域利用源时间序列,即使具有不同的特性。在各个领域的广泛实验表明,我们提出的方法在合成和真实世界数据集上的性能优于最先进的基线,消融研究验证了我们设计选择的有效性。

英文摘要:

Recently, deep neural networks have gained increasing popularity in the field of time series forecasting. A primary reason for their success is their ability to effectively capture complex temporal dynamics across multiple related time series. The advantages of these deep forecasters only start to emerge in the presence of a sufficient amount of data. This poses a challenge for typical forecasting problems in practice, where there is a limited number of time series or observations per time series, or both. To cope with this data scarcity issue, we propose a novel domain adaptation framework, Domain Adaptation Forecaster (DAF). DAF leverages statistical strengths from a relevant domain with abundant data samples (source) to improve the performance on the domain of interest with limited data (target). In particular, we use an attention-based shared module with a domain discriminator across domains and private modules for individual domains. We induce domain-invariant latent features (queries and keys) and retrain domain-specific features (values) simultaneously to enable joint training of forecasters on source and target domains. A main insight is that our design of aligning keys allows the target domain to leverage source time series even with different characteristics. Extensive experiments on various domains demonstrate that our proposed method outperforms state-of-the-art baselines on synthetic and real-world datasets, and ablation studies verify the effectiveness of our design choices.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑