UnitNorm：重新思考时间序列中Transformer的归一化
UnitNorm: Rethinking Normalization for Transformers in Time Series
AI总结：
针对时间序列Transformer中传统归一化导致的问题，提出UnitNorm方法，通过范数缩放和注意力调节，在6个模型和10个数据集上显著提升预测与分类性能。
摘要：
归一化技术对于提升Transformer模型在时间序列分析任务中的性能和稳定性至关重要，然而传统的批归一化和层归一化等方法常常导致诸如token偏移、注意力偏移和稀疏注意力等问题。我们提出了UnitNorm，一种新颖的方法，通过输入向量的范数进行缩放并调节注意力模式，有效规避了这些挑战。基于现有的归一化框架，UnitNorm的有效性在多种时间序列分析任务中得到了验证，包括预测、分类和异常检测，通过对6个最先进的模型和10个数据集的严格评估。值得注意的是，UnitNorm表现出优越的性能，尤其是在需要稳健注意力机制和上下文理解的场景中，体现在预测任务中MSE最高降低1.46，分类任务中准确率提升4.89%。这项工作不仅呼吁重新评估时间序列Transformer中的归一化策略，也为提升模型性能和稳定性指明了新方向。源代码可在https://anonymous.4open.science/r/UnitNorm-5B84获取。
英文摘要：
Normalization techniques are crucial for enhancing Transformer models' performance and stability in time series analysis tasks, yet traditional methods like batch and layer normalization often lead to issues such as token shift, attention shift, and sparse attention. We propose UnitNorm, a novel approach that scales input vectors by their norms and modulates attention patterns, effectively circumventing these challenges. Grounded in existing normalization frameworks, UnitNorm's effectiveness is demonstrated across diverse time series analysis tasks, including forecasting, classification, and anomaly detection, via a rigorous evaluation on 6 state-of-the-art models and 10 datasets. Notably, UnitNorm shows superior performance, especially in scenarios requiring robust attention mechanisms and contextual comprehension, evidenced by significant improvements by up to a 1.46 decrease in MSE for forecasting, and a 4.89% increase in accuracy for classification. This work not only calls for a reevaluation of normalization strategies in time series Transformers but also sets a new direction for enhancing model performance and stability. The source code is available at https://anonymous.4open.science/r/UnitNorm-5B84.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
