发表机构
Shanghai Jiao Tong University; University of Michigan, Ann Arbor; University of California, Berkeley; Cornell University; Technische Universität Dresden; Carnegie Mellon University; The University of Texas at Austin; University of Wisconsin–Madison（上海交通大学; 密歇根大学安娜堡分校; 加利福尼亚大学伯克利分校; 康奈尔大学; 德累斯顿工业大学; 卡内基梅隆大学; 德克萨斯大学奥斯汀分校; 威斯康星大学麦迪逊分校）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对交通流量预测中自注意力机制扩展性有限的问题，提出EMAGN，通过学习聚类矩阵将空间注意力机制线性化，降低复杂度，实验表明其在准确性和效率上优于其他模型，扩展了可行模型配置。
AI 中文摘要
交通流量预测因复杂的时空依赖性而极具挑战。自注意力机制虽被广泛采用以建模动态和长距离依赖性并取得了先进性能，但因二次计算和内存复杂度而扩展性有限。为此，我们提出了高效多注意力图网络（EMAGN），受快速高维高斯滤波理论启发，将空间注意力机制线性化。两个学习聚类矩阵C_k和C_v将键值向量自适应分组为M个超级聚类，把复杂度从O(N^2 d)降至O(NMd)，且不牺牲动态依赖性建模的注意力灵活性。在PEMS - BAY和METR - LA上的实验结果表明，EMAGN在MAE上比全注意力GMAN低2.7 - 3.2%，同时训练时间减少32%，推理时间减少38%，GPU内存减少58%。在K = 16个注意力头时，全注意力GMAN在标准11GB GPU上完全耗尽内存，而EMAGN仍可运行，展示了可行模型配置的显著扩展。由于其交通网络感知的自适应聚类，EMAGN在相同骨干网络下在准确性和效率上也超过了Linformer和Performer。
英文摘要
Traffic forecasting is highly challenging due to complex and nonlinear spatial and temporal dependencies. Self-attention mechanisms have been widely adopted to model dynamic and long-range dependencies, achieving state-of-the-art performance, but suffer from limited scalability due to quadratic computational and memory complexity. To address this, we propose an Efficient Multi-Attention Graph Network (EMAGN) that linearises the spatial attention mechanism itself, inspired by the theory of fast high-dimensional Gaussian filtering. Two learned clustering matrices C_k and C_v adaptively group key and value vectors into M super-clusters, reducing complexity from O(N^2 d) to O(NMd) without sacrificing the flexibility of attention for dynamic dependency modelling. Experimental results on PEMS-BAY and METR-LA show that EMAGN achieves accuracy within 2.7-3.2% MAE of full-attention GMAN while reducing training time by 32%, inference time by 38%, and GPU memory by 58%. Critically, at K=16 attention heads, full-attention GMAN runs out of memory on a standard 11 GB GPU entirely while EMAGN continues to operate, demonstrating a categorical expansion of feasible model configurations. EMAGN also surpasses Linformer and Performer in both accuracy and efficiency within the same backbone, owing to its traffic-network-aware adaptive clustering.
