arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2605.11287cs.LGcs.AI
收藏

超越相似性:时间序列分析中的时序操作注意力

Beyond Similarity: Temporal Operator Attention for Time Series Analysis

Georgia Institute of Technology(佐治亚理工学院)

Jevon Twitty, Vinh Pham, Nitiwith Rotchanarak, Viresh Pati, Yubin Kim, Shihao Yang, Jiecheng Lu

AI总结:

本文提出时序操作注意力(TOA),通过引入可学习的操作符增强注意力机制,以更有效地处理时间序列数据中的符号和振荡变换,提升时间序列预测、异常检测和分类任务的性能。

摘要:

时间序列预测中存在一个持久性悖论:结构简单的MLP和线性模型往往优于高容量的Transformer。我们指出,这种差距源于序列建模基本原理的不匹配:尽管许多时间序列动态由全局时间操作符(如滤波和谐波结构)主导,标准注意力将每个输出视为输入的凸组合。这限制了其表示带符号和振荡变换的能力,这些能力对于时间信号处理至关重要。我们正式将这一限制定义为softmax注意力中的简单约束混合瓶颈,这对由操作符驱动的时间序列任务尤其限制性。为了解决这一问题,我们提出时序操作注意力(TOA),一种通过显式、可学习的序列空间操作符增强注意力的框架,使时间内的符号混合成为可能,同时保持输入依赖的适应性。为了使密集的N×N操作符实用化,我们引入了随机操作符正则化,一种高方差的dropout机制,它稳定了训练并防止了记忆性学习。在预测、异常检测和分类基准上,TOA在集成到标准骨干如PatchTST和iTransformer时始终提高了性能,尤其是在重建密集任务中表现尤为突出。这些结果表明,显式操作符学习是有效时间序列建模的关键要素。

英文摘要:

A persistent paradox in time-series forecasting is that structurally simple MLP and linear models often outperform high-capacity Transformers. We argue that this gap arises from a mismatch in the sequence-modeling primitive: while many time-series dynamics are governed by global temporal operators (e.g., filtering and harmonic structure), standard attention forms each output as a convex combination of inputs. This restricts its ability to represent signed and oscillatory transformations that are fundamental to temporal signal processing. We formalize this limitation as a simplex-constrained mixing bottleneck in softmax attention, which becomes especially restrictive for operator-driven time-series tasks. To address this, we propose $\textbf{Temporal Operator Attention (TOA)}$, a framework that augments attention with explicit, learnable sequence-space operators, enabling direct signed mixing across time while preserving input-dependent adaptivity. To make dense $N \times N$ operators practical, we introduce Stochastic Operator Regularization, a high-variance dropout mechanism that stabilizes training and prevents trivial memorization. Across forecasting, anomaly detection, and classification benchmarks, TOA consistently improves performance when integrated into standard backbones such as PatchTST and iTransformer, with particularly strong gains in reconstruction-heavy tasks. These results suggest that explicit operator learning is a key ingredient for effective time-series modeling.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑