近期偏倚的因果注意力用于时间序列预测
Recency Biased Causal Attention for Time-series Forecasting
AI总结：
本文提出一种引入近期偏倚的因果注意力机制，通过平滑的重尾衰减重新加权注意力分数，增强局部时间依赖性，提升时间序列预测性能。
摘要：
近期偏倚是一种有用的归纳先验，用于序列建模：它强调附近的观测，并仍允许更长范围的依赖关系。标准Transformer注意力缺乏这一特性，依赖于所有到所有交互，忽略了时间数据的因果和通常局部结构。我们提出了一种简单的机制，通过平滑的重尾衰减重新加权注意力分数来引入近期偏倚。这种调整在不牺牲捕捉更广泛和数据特定相关性的灵活性的同时，加强了局部时间依赖性。我们证明，具有近期偏倚的注意力能够一致地提高序列建模效果，使Transformer更接近RNN的读、忽略和写操作。最后，我们展示了我们的方法在具有挑战性的时间序列预测基准上实现了竞争甚至更优的性能。
英文摘要：
Recency bias is a useful inductive prior for sequential modeling: it emphasizes nearby observations and can still allow longer-range dependencies. Standard Transformer attention lacks this property, relying on all-to-all interactions that overlook the causal and often local structure of temporal data. We propose a simple mechanism to introduce recency bias by reweighting attention scores with a smooth heavy-tailed decay. This adjustment strengthens local temporal dependencies without sacrificing the flexibility to capture broader and data-specific correlations. We show that recency-biased attention consistently improves sequential modeling, aligning Transformer more closely with the read, ignore, and write operations of RNNs. Finally, we demonstrate that our approach achieves competitive and often superior performance on challenging time-series forecasting benchmarks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
