arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2006.08849cs.IRcs.LG
收藏

保持动态注意力的长期时空预测

Preserving Dynamic Attention for Long-Term Spatial-Temporal Prediction

University of Macau(澳门大学);Joint AI and Future Network Research Institute, BNU (Zhuhai) & UIC(北京师范大学(珠海)-北师港浸大联合人工智能与未来网络研究院);Shanghai Jiaotong University(上海交通大学)

Haoxing Lin, Rufan Bai, Weijia Jia, Xinyu Yang, Yongjian You

摘要:

城市数据采掘系统越来越需要有效的长期预测。许多实际应用,如事故预防和资源预分配,需要较长的准备时间。然而,挑战来了,因为长期预测是高度误差敏感的,这在预测具有复杂和动态时空相关性的城市现象时变得更加关键。具体来说,由于有价值的相关性的数量是有限的,大量的不相关特征会引入噪声,从而引发增加的预测误差。此外,在每个时间步之后,误差都可以穿过相关性并到达每次未来预测的时空位置,从而导致显着的误差传播。为了解决这些问题,我们提出了一个动态切换注意力网络(DSAN)与一个新的多空间注意力(MSA)机制,明确衡量输入和输出之间的相关性。为了滤除不相关的噪声并减轻错误传播,DSAN通过在有噪声的输入上应用自注意来动态地提取有价值的信息,并且通过实现开关-注意力机制将每个输出直接桥接到净化的输入。通过对两个时空预测任务的大量实验,我们证明了DSAN在短期和长期预测方面的优越性。

英文摘要:

Effective long-term predictions have been increasingly demanded in urban-wise data mining systems. Many practical applications, such as accident prevention and resource pre-allocation, require an extended period for preparation. However, challenges come as long-term prediction is highly error-sensitive, which becomes more critical when predicting urban-wise phenomena with complicated and dynamic spatial-temporal correlation. Specifically, since the amount of valuable correlation is limited, enormous irrelevant features introduce noises that trigger increased prediction errors. Besides, after each time step, the errors can traverse through the correlations and reach the spatial-temporal positions in every future prediction, leading to significant error propagation. To address these issues, we propose a Dynamic Switch-Attention Network (DSAN) with a novel Multi-Space Attention (MSA) mechanism that measures the correlations between inputs and outputs explicitly. To filter out irrelevant noises and alleviate the error propagation, DSAN dynamically extracts valuable information by applying self-attention over the noisy input and bridges each output directly to the purified inputs via implementing a switch-attention mechanism. Through extensive experiments on two spatial-temporal prediction tasks, we demonstrate the superior advantage of DSAN in both short-term and long-term predictions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑