arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2605.18534cs.LG
收藏

XCTFormer: 利用跨通道和跨时间依赖性提升时间序列分析

XCTFormer: Leveraging Cross-Channel and Cross-Time Dependencies for Enhanced Time-Series Analysis

The Stein Faculty of Computer and Information Science(施坦计算机与信息科学系);Ben-Gurion University of the Negev(本·古里安大学)

Israel Zexer, Omri Azencot

AI总结:

本文提出XCTFormer模型,通过增强的注意力机制显式捕捉时间序列中的跨时间与跨通道依赖性,以提升时间序列分析性能,特别是在缺失值填补任务中取得state-of-the-art结果。

摘要:

多变量时间序列分析涉及从多个相互依赖变量的序列中提取信息性表示,支持预测、填补和异常检测等任务。在现实场景中,这些变量通常来自共享上下文或底层现象,表明存在时间与通道间的潜在依赖性,可以利用以提高性能。然而,最近的研究发现,假设无变量间依赖性的通道独立(CI)模型往往优于显式建模此类关系的通道依赖(CD)模型。这一意外结果表明,当前CD模型可能由于依赖性捕捉的限制而未能充分发挥潜力。最近的研究重新审视了通道依赖建模,但这些方法通常采用间接建模策略,可能导致有意义的依赖性被忽视。为了解决这个问题,我们引入了XCTFormer,一种基于Transformer的通道依赖(CD)模型,通过增强的注意力机制显式捕捉跨时间和跨通道依赖性。该模型以token到token的方式操作,建模时间与通道之间每对token之间的成对依赖性。架构包括(i)数据处理模块,(ii)新型的跨关系注意力块(CRAB),以增加容量和表达性,以及(iii)可选的依赖压缩插件(DeCoP),以提高可扩展性。通过在三个时间序列基准上的广泛实验,我们证明XCTFormer在与广泛认可的基线相比时取得了强劲的结果;特别是,在填补任务中,它在MSE和MAE上分别比第二好的方法平均高出20.8%和15.3%。

英文摘要:

Multivariate time-series analysis involves extracting informative representations from sequences of multiple interdependent variables, supporting tasks such as forecasting, imputation, and anomaly detection. In real-world scenarios, these variables are typically collected from a shared context or underlying phenomenon, suggesting the presence of latent dependencies across time and channels that can be leveraged to improve performance. However, recent findings show that channel-independent (CI) models, which assume no inter-variable dependencies, often outperform channel-dependent (CD) models that explicitly model such relationships. This surprising result indicates that current CD models may not fully exploit their potential due to limitations in how dependencies are captured. Recent studies have revisited channel dependence modeling with various approaches; however, these methods often employ indirect modeling strategies, which can lead to meaningful dependencies being overlooked. To address this issue, we introduce XCTFormer, a transformer-based channel-dependent (CD) model that explicitly captures cross-temporal and cross-channel dependencies via an enhanced attention mechanism. The model operates in a token-to-token fashion, modeling pairwise dependencies between every pair of tokens across time and channels. The architecture comprises (i) a data processing module, (ii) a novel Cross-Relational Attention Block (CRAB) that increases capacity and expressiveness, and (iii) an optional Dependency Compression Plugin (DeCoP) that improves scalability. Through extensive experiments on three time-series benchmarks, we show that XCTFormer achieves strong results compared to widely recognized baselines; in particular, it attains state-of-the-art performance on the imputation task, outperforming the second-best method by an average of 20.8% in MSE and 15.3% in MAE.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑